There are several ways to formalize the relationship between the distances and co-occurrence statistics, in this paper we use the following: x,y where x,y is the squared distance between the embeddings of x and y,p(x) andp(y) are empirical probabilities, and Z = x,yp (x)p(y)e −d 2 x,y is a normalization term.
section 3 describes the dataset and the construction of the substitute vectors.