Fluency ratings can be carried out without the presence of a reference translation, reducing any remnant bias towards reference translations in the evaluation setup.
we propose a judgment regime in which each task is presented as a two-item fluency and adequacy judgment, evaluated separately, and with adequacy restructured into a monolingual "similarity of meaning" task.