For this purpose, we used the trained Conditional Random Field (CRF) model (see (Liakata and others, 2012)) to automatically assign CoreSC concepts to the CRA corpus and evaluated the results on the gold standard.
this was not followed strictly by all annotators leading to some variation in the number of annotated sentences across the different annotators.