We use the average empirical variance of the log posterior probabilities of the rules among the learned grammars over a held-out set S as a proxy of the diversity among the grammars: where R(G, s) represents the set of rules extracted from the chart when parsing sentence s with grammar G, and VAR(log(p(r|s, G))) is the variance of log(p(r|s, G)) among all grammars G ∈ G. Note that the average empirical variance is only an approximation of the diversity among grammars.
this measure tends to be biased to produce larger numbers when the posterior probabilities of rules tend to be small, because small differences in probability produce large changes in the log scale.