However, the MMI principle is applied only at the testing phase rather than the training phase.
it can only be used to evaluate the quality of a generation by estimating its mutual information with the context.