Let I = I + I − be the index set of S, |I ⇔ n2 where from (10) to (11) we expand the sums and reorder the terms.
we obtain the well-order loss Lwell-order(Sms, Spr) = where S ms , S pr are the scores for (mention, subject), (pattern, relation) pairs for a question, I, J are the index sets for candidate subjects, relations in the ranked subgraph, +, − indicate the correct candidate and incorrect candidate, [•] + = max(•, 0), and λ > 0 is a margin term.