Among diverse methods for calculating word embeddings, we empirically show the uncorrelation in word embedding dimensions, which is one of the prerequisites for simplifying the bilinear operator to a linear one.
the second expectation term in the right hand side of (18) can be computed as follows: When i = j, each of the four expectations in the RHS of (21) are zero from the uncorrelation assumption.