To quantify the difference of two corefer-ence system outputs S 1 and S 2 , given a key K, we count how many of the mentions m are classified differently using a mention classification function c: The mention classification function c requires a class inventory which is not featured by the common evaluation metrics for coreference resolution.
10 Therefore, we adapt the mention classification paradigm introduced in the ARCS framework for coreference resolution evaluation (Tuggener, 2014) which assigns one of the following four classes to a mention m given a key K and a system response S: one issue with ARCS is to determine a criterion for the TP class, i.e.