Neural machine translation (NMT) has achieved the state-of-the-art results on a mass of language pairs with varying structural differences, such as English-French (Bahdanau et al., 2014;Vaswani et al., 2017) and Chinese-English (Hassan et al., 2018).
so far not much is known about how and why NMT works, which pose great challenges for debugging NMT models and designing optimal architectures.