For the implementation of this approach, we chose a problem in the sentiment annotation domain, where diverse judgments representative of several communities are valuable due to the subjective nature of evaluating sentiment in a given context.
we chose to collect judgments on the toxicity of Wikipedia discussion comments, where a "toxic" comment is defined as any kind of hateful, aggressive, or disrespectful comment that is likely to make someone leave a discussion.