A better mathematical grounding of the distillation method's apparent heuristic in terms of existing probabilistic models sheds light on the mixed performance of distillation across languages and experimental settings.
it turns out that the distillation method of DLD09 is equivalent to one iteration of EM prior re-estimation in a Naïve Bayes model.