We also experiment with ELMO trained only on in-domain data as an example of high-resource LM pretraining methods, such as Dai and Le (2015), when there is no out-of-domain data available.
we generate contextual word representations with a Transformer-based ELMO.