For this, we need large corpora of cloze questions C and natural questions Q. Cloze Corpus We create the cloze corpus C by applying the procedure outlined in Section 2.2.2.
we consider Noun Phrase (NP) and Named Entity mention (NE) answer spans, and cloze question boundaries set either by the sentence or sub-clause that contains the answer.