Although sequencelevel smoothing is computationally efficient compared to reinforcement learning approaches (Ranzato et al., 2016;Rennie et al., 2017), it is slower compared to MLE.
we need to forward each of the samples y l through the RNN in teacher-forcing mode so as to compute its hidden states h l t , which are used to compute the sequence MLE loss as To speed up training, and since we already forward the ground truth sequence in the RNN to evaluate the MLE part of α Seq (y * , x), we propose to use the same hidden states h * t to compute both the MLE and the sequence-level smoothed loss.