(2016), we incorporate the action result o i into the inputv i of LSTM p at time-step i and compute hidden state h i ∈ R d of word x i as: where  , which could be cast as a binary classification problem.
we use a logistic function to compute where θ r = {W r ∈ R 1×4d , b r ∈ R} is the parameter to be learned.