A review of the literature shows that theoretical studies on annotator bias are not convergent.
opposite arguments have been proposed concerning Cohen's κ (Di Eugenio and Glass 2004, Arstein and Poesio 2008, Hayes, 2007.