It is important to note that crowdsourced evaluations can lead to additional noise compared with standard lab-based evaluation, mostly due to the possibility of uncooperative evaluators.
the randomization of the order of the evaluated utterances ensures that such noise does not bias the results towards one system.