However, as explained in the introduction, most gradients are back-propagated through attention layer to the encoder's hidden state h t , not directly to the final memory state, and thus provide little incentive for the encoder to memorize salient information in c t .
to enhance encoder's memory, we add a closed-book decoder, which is a unidirectional LSTM decoder without attention/pointer layer.