Therefore, we design a more implicit loss to help the student refrain from the incoherent translation results by acting towards the teacher in the hidden-state level: where , and φ is a penalty function.
we let where −1 ≤ γ st , γ tr ≤ 1 are two thresholds controlling whether to penalize or not.