Finally, the inability of the M 2 Scorer to combine corrections from different annotators (as opposed to selecting only one annotator's corrections for the whole sentence) can also result in underestimations of performance.
it is clear that exploring these combinations during evaluation is a challenging task itself.