(2017)) use to compare models focus on sentiment, entailment, and similarity.
models like BERT (Devlin et al., 2018) are trained to predict random missing words in very large corpora (see Table 1 for the data requirements of the models we compare).