This selection was likely made to avoid the noise of learning multiple senses for infrequent words.
our method is robust to noise, which can be seen by the good performance of our model that learns multiple embeddings for the top 30,000 most frequent words.