To compose the model, both the encoder and decoder are implemented employing RNNs, i.e., the encoder converts source words into a sequence of vectors, and the decoder generates target words one-by-one based on the conditional probability shown in Equation (3).
the encoder takes a sequence of source words as inputs and returns forward hidden vectors − → h j (1 ≤ j ≤ J) of the forward-RNN: Similarly, backward hidden vectors ← − h j (1 ≤ j ≤ J) of the backward-RNN are obtained, in reverse order.