Let D be a dataset that consists of triples including a posted question q, a headline candidate x, and a voted score y, i.e., (q, x, y) ∈ D. In the pairwise ranking method, we train a ranking model as a binary classifier that determines whether the condition y i > y j is true or false for two candidates x i and x j in the same question (q i = q j ).
we defined it with respect to the best candidate, i.e., precision@1, as follows: No.