relevant (P ∪ U) and irrelevant (N ), as was done in SemEval.
by ignoring the difference in utility between paraphrases and useful questions during training, a binary classification approach is likely to underperform a ranking approach that is trained on all the ranking triples implied by the original 3 categories of questions.