Early studies on recurrent neural network (RNN)-based model analyze the translation quality with respect to the sentence length, and show that their models improve translations for long sentences, using the long short-term memory (LSTM) (Sutskever et al., 2014) or introducing the attention mechanism (Bahdanau et al., 2015;Luong et al., 2015).
koehn and knowles (2017) report that even RNN-based model with the attention mechanism performs worse than phrase-based statistical machine translation (koehn et al., 2007) in translating very long sentences, which challenges us to develop an NMT model that is robust to long sentences or more generally, variations in input length.