(2018); Domhan (2018) introduce the techniques in Transformers into RNN-based models, with reportedly higher BLEU.
it would be interesting to see if the same result holds true with their architectures.