To simplify the analysis, we further assume that X l = 1 ∀l.
the similarity between two words i, j can be plausibly measured (as customarily done) by the cosine distance between their contextual represen- the average distance is not equivalent to the distance between the average representations alone, but has to include an additional bias term cov(X i , X j ) -which reflects the statistical dependence between the two vector representations X i , X j .