Each word x i is represented by concatenating the forward hidden state − → h i and the backward one In this way, the source sentence X = {x 1 , ..., x Tx } can be represented as annotations H = {h 1 , ..., h Tx }.
in the decoder, an RNN hidden state s j for time j is computed by: The context vector c j is then, computed as a weighted sum of these annotations H = {h 1 , ..., h Tx }, by using alignment weight α ji : 3 Sentence Embedding and Selection A source sentence can be represented as the annotations H. the length of H depends on the sentence length T x .