We trained monolingual ELMo and FastText with default parameters.
for monolingual training, we used the 1 Billion Word benchmark (Chelba et al., 2014) for English, and equivalent subsets of ∼400 million tokens from WMT'13 (Bojar et al., 2013) news crawl data.