Transfer learning (Pan and Yang, 2010) has been successfully applied to numerous domains in ma-chine learning, such as machine translation (Zoph et al., 2016), computer vision, (Sharif Razavian et al., 2014), and speech recognition (Doulaty et al., 2015).
the rise of large-scale human annotated datasets with over 100,000 realistic question-answer pairs such as SQuAD (Rajpurkar et al., 2016), NewsQA (trischler et al., 2016), and MSMARCO (Nguyen et al., 2016), has led to a large number of successful deep learning models (Lee et al., 2016;Seo et al., 2017;Dhingra et al., 2017;Wang and Jiang, 2016).