The issue is more important when dealing with small-to-moderate data sets.
even for a 130K test set (Sections 22-24 of the Wall Street Journal corpus, standardly used as a test set in POS-tagging benchmarks), it is useful to know the estimated noise rate, as it is not clear that all reported improvements in performance would come out significant.