The tools used for corpora annotation offer to calculate some interannotator agreement statistics.
the WebAnno we are using, makes it possible to calculate Kohen's Kappa, Fleiss' Kappa and Krippendorff's Alpha.