Contextualized word embeddings have been used across a broad range of NLP tasks, outperforming the previous state-of-the-art models.
several works showed that they capture various types of linguistic knowledge, from syntactic to semantic and discourse relations (e.g.