First, we assign a true/false label to each question-choice pair using the standalone QA system along with an associated confidence score s 1 .
we identify the natural language relation (entailment, contradiction or neutral) between each ordered pair of choices for a given question, along with an associated confidence score s 2 .