Although the introduction of RNNs as a decoder has resulted in substantial improvements in terms of translation quality, simultaneously it imposes a serious restriction on the capability of encoder-decoder framework caused by the structure of RNNs.
when the RNN decoder generates the t-th word y t in decoding phase, only y <t can be utilized, while the possible words y >t are directly neglected.