Although MT may be used in solving this task, it is only used by the algorithms -the final evaluation is done in the source language.
all three annotators were native English speakers who were not system developers for any of the 5W systems that were being evaluated (to avoid biased grading, or assigning more blame to the MT system).