KL-divergence is a natural measure of discrepancy between two distributions, but it is somewhat nontrivial to compute-for our three recursive models, it requires solving fixed point equations, and becomes completely intractable in face of label symmetry.
we propose a more manageable alternative: where we weight the difference between the j-th component of the parameter vectors by µ j , the jth expected sufficient statistic with respect to p θ (the expected counts computed in the E-step).