One way to achieve this is by simply representing each mention via the sum of the mentions currently in its coreference cluster.
formally, let C(i) be a coreference cluster (as defined by the antecedent distribution P (y i )) such that i ∈ C(i), and replace Equation 1 with: each span will now contain information about all of its current coreference cluster, effectively equalizing the representations of different spans belonging to the same cluster.