To extend this approach, in Figure 1 agreement measures are provided 1 for lemmas, full POS tags, main tags only, agreement on single features instead of feature sets, and agree-1 Originally, the tagging task was assigned to a pool of nine annotators, each of which was assigned an individual set of documents as well as a selection of documents from the set of the previous annotator for the measurement of inter-annotator agreement.
one of the annotators (annotator e) left the pool, which means that the agreement for the pairs d-e and e-f could not be measured, resulting in a total of seven pairs in Figure 1.