Such networks employ two (sometimes identical) RNNs, one for encoding (i.e., generating H (x) ) and another for decoding.
for decoding y j , the decoder RNN takes the embedding y j−1 to generate a hidden vector h j as a single vector which is mapped into an output space for a sof tmax operation (Eq.