Our interest being in difficult, disagreeable cases, we group the instances into 5 bands according to the observed level of disagreement and the tendency in the majority of the annotations.
items with at most two label 1 annotations are clearly semantically new, while those with at least 17 (out of 20) are clearly semantically old.