Some manual measures ask annotators to explicitly mark errors, but this has been found to have even lower agreement than ranking (Lommel et al., 2014).
while providing the gold standard for MT evaluation, human evaluation is not a scalable solution.