Neural Machine Translation (NMT) is an endto-end learning approach to machine transla-tion which has recently shown promising results on multiple language pairs (Luong et al., 2015;Shen et al., 2015;Wu et al., 2016;Tu et al., 2016;Zhang and Zong, 2016;Jean et al., 2015;Meng et al., 2015).
bi-directional RNNs are proposed to integrate information from the past and the future.