• F is neutral if it treats all items symmetrically, i.e., if for every two items j and j with the same set of possible categories (i.e., with C j = C j ) and for every profile A, it is the case that whenever N A j:a = N A j :a for all answers a ∈ A j = A j , then F (A) j = F (A) j .
if the patterns of individual annotations of j and j are the same, then also their collective annotation should coincide.