Let D be a dataset that consists of triples including a posted question q, a headline candidate x, and a voted score y, i.e., (q, x, y) ∈ D. In the pairwise ranking method, we train a ranking model as a binary classifier that determines whether the condition y i > y j is true or false for two candidates x i and x j in the same question (q i = q j ).
we first define the index pairs of positive examples by P Note that we do not need to consider the negative examples N = {(j, i) | (i, j) ∈ P } since they yield the same formula as P in the optimization process.