If multiple ref-erences were used that Bleu would likely have stronger correlation.
it is clear that the cost of hiring professional translators to create multiple references for the 2000 sentence test set would be much greater than the $10 cost of collecting manual judgments on Mechanical Turk.