To address this, we propose teacher annealing, which mixes the teacher prediction with the gold label during training.
the term in the summation becomes where λ is linearly increased from 0 to 1 throughout training.