To train NMT1, NMT2 and SA-NMT, we employed the same settings for fair comparison.
except the stopping iteration which was selected using development data, we used the default settings set out in (Bahdanau et al., 2015) for all NMT-based systems: the dimension of word embedding was 620, the dimension of hidden units was 1000, the batch size was 80, the source and target side vocabulary sizes were 30000, the maximum sequence length was 50, 6 the beam size for decoding was 12, and the optimization was done by Adadelta with all hyper-parameters suggested by (Zeiler, 2012).