This seems particularly appropriate when dealing with uncooperative annotators, as might be encountered, for example, in crowdsourcing (Snow et al., 2008;Zhang et al., 2016).
with a team of trained annotators, we believe that honest disagreements could contain valuable information better not ignored.