Levy and Goldberg (2014) showed that the function achieves its global maximal value when for each word-pair (x, y) the inner product of the embedding vectors x • y is equal to pmi(x, y).
they showed that S(m) ≤ S(pmi) for every matrix m. We next show that the value of the function S(m) at its maximum point, the PMI matrix, has a concrete interpretation, namely it is exactly the Jensen-Shannon Mutual Information (JSMI) between words and their contexts.