We also evaluate Turker translation quality by using them as reference sets to score various submissions to the NIST MT evaluation.
we measure the correlation (using Pearson's r) between BLEU scores of MT systems measured against nonprofessional translations, and BLEU scores measured against professional translations.