We thus also implement Variational-Bayes (VB) training with symmetric Beta priors on θ j and symmetric Dirichlet priors on the strategy parameters, ξ j .
all interannotator agreement measures suffer from an even more fundamental problem: removing/ignoring annotators with low agreement will always improve the overall score, irrespective of the quality of their annotations.