We showed that the optimization of skip-gram embeddings with negative sampling finds the best low-dimensional approximation of the JSMI measure.
there are several standard methods of measuring the distance between two discrete probability distributions, defined on a given finite set A.