Indeed, both are trying to assess the quality of the translation output, whether it is produced by a computer or by a foreign language student.
one has to take care not to bias towards the correct response so strongly that the student is allowed to make mistakes with impunity.