The final context dependent token representation h (e) t is the concatenation of the forward and backward pass token representations: To obtain the final context dependent token representation c j at the decoding time step j, we take a weighted average over to- Bahdanau et al.
encoder: We use a bi-directional LSTM (Graves et al., 2005) with attention mechanism as our sentence encoder.