(2011), Lee and Lin (2014)), described as follows.
for each sentence x s , we define the set of preference pairs as the set of ordered translation pairs for which the evaluation score prefers the first element: following Lee and Lin (2014), we define the loss (or, error) of any sentence s as the sum of its pairwise squared hinge losses: no loss is contributed by preference pairs for which the classification score correctly prefers the first element by a large-enough margin, i.e., h s i ≥ h s j +1; all other preference pairs contribute some loss.