The decoder is essentially a neural language model conditioned on the input sentence representation h N i .
two RNNs are used (one for the sentence s i−1 and the other for the sentence s i+1 ) with different parameters except the embedding matrix (E), and a new set of matrices (C r , C z and C) are introduced to condition the GRU on h N i .