It depends on what is the natural degree of agreement over subtle connotation among people.
we also report the degree of agreement among human judges in Table 7, where we compute the agreement of one Turker with respect to the gold standard drawn from the rest of the Turkers, and take the average across over all five Turkers 18 .