The last setup step was to assign a quality score to the translations.
in line with other approaches to quality (Callison-Burch et al., 2007), we defined a scale of quality that distinguishes between 4 comprehension levels: Q1) no errors or only minor ones that do not affect comprehension; Q2) comprehension problems, but most of the original meaning is still preserved; Q3) severe errors, but some meaning can still be inferred; and finally, Q4) meaning is completely compromised 8 .