We are concerned here with the construction of datasets and evaluation of systems within natural language processing (NLP).
humans providing responses that are used to derive graded values on natural language contexts, or in the ordering of systems corresponding to their perceived performance on some task.