A study of the inter-annotator agreement between two human annotators has been performed on a set of 100 questions.
(Table 1), assuming the system only tagged operas as AP, lenient accuracy will be 1, exact accuracy will be 0, precision for the AskingPoint class will be 1 and its recall will be 0.5.