When re-translating the source sentence, for each partially expanded decoding path, we compute the ratio of word pairs that satisfy such ordering constraints 2 .
given a partially expanded path is covered by a full decoding path Q (from other system outputs), we denote the relationship as (s i < s j ) ∈ Q.