By choosing perceived offensiveness and discrepancies in perception as proxy measures for MAS, we hope to shift studies of MAS from a judgment call on an individual's intent to a description of how it affects its targets, which can be described as an objective measure of human behavior that can be validated through study.
this moderate agreement is on-par with other difficult annotation tasks, such as those for connotation frames (Rashkin et al., 2016), which had 0.52 percentage agreement for rating the polarity of the sentence towards a target, dimensions of social relationships (Rashid and Blanco, 2017), which had κ val- ues as low as 0.59, and Word Sense Disambiguation (Passonneau et al., 2012), which reported α values for some words below 0.30 at determining meaning.