We found that the overall system ranking remained largely unchanged for both shared tasks, except for some of the lower ranking systems that changed one or two places.
there was a considerable drop in the magnitude of all B 3 scores owing to the combination of two things: (i) mention manipulation, as proposed by Cai and Strube (2010), adds singletons to account for twinless mentions; and (ii) the B 3 metric allows an entity to be used more than once as pointed out by Luo (2005).