In an evaluation on four tasks, we show that attentive mimicking outperforms previous work for both rare and medium-frequency words.
compared to previous work, attentive mimicking improves embeddings for a much larger part of the vocabulary, including the mediumfrequency range.