When comparing word pairs similarities obtained from word embeddings, to word pairs similarities obtained from human judgement, it is ob-served that word embeddings slightly underestimate the similarity between similar words, and overestimate the similarity between distant words.
for example, in the WS353 (finkelstein et al., 2001) word similarity ground truth: sim("shore", "woodland") = 3.08 < sim("physics", "proton") = 8.12 the use of GloVe 42B 300d embedding with cosine similarity (see Section 4) yields the opposite order: sim("shore", "woodland") = 0.36 > sim("physics", "proton") = 0.33 Re-embedding the space using a manifold learning stage can rectify this.