The question addressed in this article is: How problematic is learning from training data with annotation noise?
we are interested in estimating the degree to which performance on easy instances at test time can be hurt by the presence of hard instances in training data.