They suggest that for certain error annotation tasks, such as preposition usage, it may not be appropriate to use only one rater and that if one uses multiple raters for error annotation, there is the possibility of creating an adjudicated set, or at least calculating the variability of the system's performance.
annotation with multiple raters has its own disadvantages as it is much more expensive and timeconsuming.