Therefore in bi-directional RNNs, not only a history vector of word w t is regarded but also a future vector.
this leads to the following conditioned probability for the history h t at time step t ∈ [1, n]: the network can be split into three parts: a forward pass which processes the original sentence word by word (Equation 1); a backward pass which processes the reversed sentence word by word (Equation 2); and a combination of both (Equation 3).