More broadly, non-expert annotation has been used extensively to gather question-answer pairs over natural language texts, for example in reading comprehension (Rajpurkar et al., 2016;Richardson et al., 2013;Nguyen et al., 2016) and visual question answering (Antol et al., 2015).
while these treat question answering as an end task, we regard it as a representation of predicateargument structure, and focus annotators on a smaller selection of text (a few target words in a single sentence, rather than a paragraph) aiming to achieve high coverage.