In real experiments, losses are averaged over all training data.
we design a more implicit loss to help the student refrain from the incoherent translation results by acting towards the teacher in the hidden-state level: where , and φ is a penalty function.