The higher compression ratio renders it more challenging for the student model to absorb important weights.
with the number of epochs increasing, the student model learned with this vanilla KD framework quickly reaches saturation on the test set (see Figure 2 in Section 4).