Even humans will struggle with sarcastic comments drawn from unfamiliar topics, for instance, obscure hobbies or art forms.
we also test human and machine performance on comments drawn solely from the politics subreddit, a topic for which all evaluators had sufficient background information.