An explanation for such behaviour may be found on the fact that long sentences are also difficult translations.
with the attention mechanism (Bahdanau et al., 2014), a decoder is used to decide which part of the source sentence to pay attention to and predict corresponding word representation at time t together with history predictions before time t. Then, a softmax is used to restore the word representation to natural target words.