For the WSsim/SemCor dataset, the correlation between original and WSsim annotation was ρ = 0.234, ρ = 0.448, and ρ = 0.390 for the three annotators, each highly significant with p < 2.2e-16.
5 Unlike previous word sense annotation projects, we asked annotators to provide judgments on the applicability of every WordNet sense of the target lemma with the instruction: 6 2 Throughout this paper, a target word is assumed to be a word in a given PoS.