For each example (x, y) ∈ D, the algorithm obtains a trajectory of states x 1 , x 2 , ..., x n visited by the current policy by running it to completion (i.e., repeatedly taking the highest scoring action until reaching an end state) from the start state x.
we instead sort the mentions in descending order by their highest scoring candidate coreference link according to the mention-ranking model.