(2) 6: until convergence Output: Distill student network p θ and teacher network q ning over multiple examples), requiring joint inference.
as mentioned above, p is more lightweight and efficient, and useful when rule evaluation is expensive or impossible at prediction time.