At the same time, they are noisy and may not reflect the desired phenomenon.
we also conducted an experiment on Amazon Mechanical Turk 6 (AMT) to obtain additional gold references, which are further required by metrics such as BLEU (Papineni et al., 2002).