While BLEU is designed to benefit from multiple reference translations (Papineni et al., 2002), this is a luxury rarely afforded to the modern system, and both of the major MT workshops use single-reference BLEU to evaluate all their tasks 34 .
many of our translations are crowdsourced, and there are no guarantees on the fluency of the participants.