We overcome this issue by leveraging recent progress in unsupervised machine translation Lample and Conneau, 2019;Artetxe et al., 2018).
we collect a large corpus of natural questions and an unaligned corpus of cloze questions, and train a seq2seq model to map between natural and cloze question domains using a combination of online back-translation and de-noising auto-encoding.