These word embeddings were learned from unsupervised Neural Language Modelling (NLM) trained on fixed-length contexts.
by recasting the same word types across different sense-inducing contexts, these representations became insensitive to the different senses of polysemous words.