Agreement: Pairwise agreement A m,n between two annotators m, n ∈ 1:J is defined as the proportion of items i ∈ 1:I for which the annotators supplied the same label, where I(s) = 1 if s is true and 0 otherwise.
a m,n is the maximum likelihood estimate of chance of agreement in a binomial model.