In recent years, there have been several strands of work which attempt to collect human-labeled data for this task -in the form of document, question and answer triples -and to learn machine learning models directly from it (Richardson et al., 2013;Berant et al., 2014;Wang et al., 2015).
these datasets consist of only hundreds of documents, as the labeled examples usually require considerable expertise and neat design, making the annotation process quite expensive.