(2016) throughout this section.
given a source sequence  Similarly, a backward RNN ← − Φ enc reads X from right to left, word by word, and generates a sequence of backward annotation vectors (1) The final annotation vector for a given time step i is the concatenation of forward and backward vec- each source sequence X is encoded into a sequence of annotation vectors , which are in turn used by the decoder: essentially a neural language model (LM) (Bengio et al., 2003) conditioned on the previously emitted words and the source sentence via an attention mechanism.