In our first bias method, we use p l (•) to bias the probability distribution calculated by the vanilla NMT model.
we add a small constant ϵ to p l (•), take the logarithm, and add this adjusted log probability to the input of the softmax as follows: We take the logarithm of p l (•) so that the values will still be in the probability domain after the softmax is calculated, and add the hyper-parameter ϵ to prevent zero probabilities from becoming −∞ after taking the log.