Part of this gap could be explained by the fact that post-editors who produced the initial post-editions in (Potet et al., 2012) had the access only to the source sentences and their automatic translations, whereas for our experiments we gave the annotators the access to all the available data: source sentences, automatic translations, post-editions and reference translations, so they could decide on the optimal labelling from a range of possibilities including existing corrections and their own knowledge.
where our phrase labelling would consider as "BAD" a phrase which has been modified in the post-edited version of the translation, a human annotator might consider the meaning as unchanged and therefore would not label this modification as an MT error.