Confidence in annotators is not modelled.
it has been extended to incorporate the reliability of the annotator with a similar method that also combines Expectation Maximization with CRF in an NER and NP chunking task (Rodrigues et al., 2014  In this paper, we presented a hybrid mention detection method combining state-of-the-art automatic mention detectors with a gamified, twoplayer interface to collect markable judgements.