Transformer shows the worst performance among the four Transformer-based models on the sentences longer than those in the training data for any controlled length.
on the shorter sentences than those in the training data, RNN-Transformer scores almost the same as Transformer on the Middle and Long training data of ASPEC English-to-Japanese and also shows a larger drop than RNN-NMT at length of -24 on the Long training data of WMT2014 English-to-German.