This baseline does not make use of various advances in NMT architectures and training tricks.
to the baseline, we use a BiDeep RNN architecture (Miceli Barone et al., 2017), label smoothing (Szegedy et al., 2016), dropout (Srivastava et al., 2014), word dropout (Sennrich et al., 2016a), layer normalization (Ba et al., 2016) and tied embeddings (Press and Wolf, 2017).