As training goes on, we gradually bias towards emulating the teacher predictions to effectively distill the structured knowledge.
we define π (t) = min{π 0 , 1 − α t } at iteration t ≥ 0, where α ≤ 1 specifies the speed of decay and π 0 < 1 is a lower bound.