Neural language models can also be used as contextualized word representations (Peters et al., 2018).
language modeling is a good benchmark task for investigating the general frameworks of neural methods in NLP field.