The key design of ELMo is to train an LSTM-based language model and utilize the hidden vector as contextualizd word embeddings.
every word is assigned a vector according to information on the whole sentence rather than several neighboring words in a limited context.