In comparison to several advanced deep NMT models, such as the Google NMT (8 layers, 24.61 tokenized BLEU)  and the LAU-connected NMT (4 layers, 23.80 tokenized BLEU) (Wang et al., 2017a), the performance of our shallow model (23.31) is competitive.
when replacing LSTM in the Google NMT with our ATR model, the GNMT+ATR system achieves a BLEU score of 24.16, merely 0.45 BLEU points lower.