Our method answers 56% of questions while maintaining 80% accuracy; in contrast, directly using the model's probabilities only answers 48% at 80% accuracy.
for each, we sampled 100 errors in which the system confidently gave a wrong answer (overconfident), and 100 errors in which the system abstained but would have gotten the question correct if it had answered (underconfident).