This allows us to have control over the variability of the experiments while answering the questions we posed above (such as the quality of the systems providing the pseudo references, the quality of MT systems being evaluated, and the diversity over the distribution of training examples).
we reserved four systems (MT2, MT5, MT6, and MT9) for the role of pseudo references.