In Section 2 we introduce some basic terminology and argue that there are four natural forms of collective annotation.
for some annotation tasks, no integrity constraints may be known to us in advance, even though we may have reasons to believe that the individual annotators do respect some such constraints.