This does not affect the ELMo component of the word embeddings.
even common words are replaced with probability at least 1 10 , making the model rely on the ELMo embeddings instead of the learned embeddings.