The distribution vector P (w) of a word w can be calculated in multiple ways.
in this work, we leverage a pretrained language model to compute P (w) and condition on all the words preceding w. for the t-th word x t in a sentence, we have where LM (w j |x <t ) denotes the probability of the j-th word in the vocabulary appearing after the sequence Note that the language model is pretrained using the same training corpus of the NMT model.