Pairs are divided into four categories by source: (1) fragments that come from different documents; (2) fragments that come from different sentences within a same document and that do not contain coreferent entity mentions; (3) fragments that come from different sentences within a same document and that contain coreferent entity mentions; (4) fragments that come from the same sentence.
this ensures that clustering the fragments back into the original corpus has a collocation score of 1.