5It is not possible to treat the missing annotations as "total disagreement" because per Krippendorff (1995), α U has no concept of this; there is no lowest disagreement score.
are lower than the agreement among extensively trained annotators reported by Stab and Gurevych (2014) (α U = 0.7726, 0.6033, 0.7594).6 they are broadly comparable to interannotator agreement scores reported in similar (and in some cases, even simpler) discourse-level argument annotation studies with expert-trained annotators, such as Aharoni et al.