Both networks share the same base model architecture and initial parameter values, but follow different training paradigms (Laine and Aila, 2017).
the student network is updated via backpropagation, then the teacher network is updated with an exponential average of the student network's parameters (Tarvainen and Valpola, 2017).