Across all documents, a perfect agreement was never achieved, which confirms the difficulty of annotating such a subjective task: The average lccm per document on the manual annotations is 0.56 (dense chains), respectively 0.54 (merged chains).
the considerable overlap between the annotators still enables us to evaluate automatic chaining methods, and the lccm agreement score serves as an upper bound.