Results with External Data As shown in Table 4, incorporation of the RACE dataset for semisupervised learning improves accuracy from 65.3% to 70.4%.
mPNet and SemimPNet still underperform a pretrained state-of-the-art neural language model One-billion-word-Lm (Jozefowicz et al., 2016).