The two mechanisms could also work together, so that both the encoder and the decoder could be improved: The previously proposed word prediction mechanism could be used only as a extra training objective, which will not be computed during the translation.
the computational complexity of our models for translation stays exactly the same.