It is a central idea of the WMTs that automatic measures of machine translation quality are an imperfect substitute for human assessments.
manual evaluation of the system outputs are conducted and their results are reported as the final rankings of the workshops.