Basically, knowledge distillation provides more information to new models for training and improves the new model's performance.
when a new model which is usually smaller is trained with the distilled knowledge from the trained deep model, it can achieve a similar (or sometimes even better) performance compared to the pretrained deep model.