It can be challenging to train multi-task neural networks that outperform or even match their single-task counterparts.
the difference with Single→Multi was not statistically significant, suggesting there is little value in multiple rounds of distillation.