Moreover, a quantitative analysis of tag confusions reveals that the majority of disagreements are due to linguistically debatable cases rather than annotation errors.
we show that even in the absence of annotation guidelines only 2% of annotator choices are linguistically unmotivated.