This is represented in contingency Table 1, where: • n ac : number of questions for which the answer is correct • n aw : number of questions for which the answer is incorrect • n u : number of questions not answered • n: number of questions (n = n ac + n aw + n u ) Let's start studying a simple utility function able to establish the preference order we want: • -1 if question receives an incorrect response • 0 if question is left unanswered • 1 if question receives a correct response Let U(i) be the utility function that returns one of the above values for a given question i.
if we want to consider n questions in the evaluation, the measure would be: The rationale of this utility function is intuitive: not answering adds no value and wrong answers add negative values.