We might however ask whether a different model could provide a similarly fitting explanation.
heterogeneity among annotators has been seen as a major source of noise in the aggregate data and there are several works attempting to separate high quality annotators from low quality ones (Raykar et al., 2009;Donmez et al., 2009;Sheng et al., 2008;Carpenter, 2008).