Last year's evaluation exercise (Lee and Przybocki, 2005) was startling in that Bleu's rankings of the Arabic-English translation systems failed to fully correspond to the manual evaluation.
the entry that was ranked 1st in the human evaluation was ranked 6th by Bleu.