2 can be rewritten as follows by a simple reformulation: Here, let us substitute e i • o j in Eq.
we can obtain the following relation for SGNS: Note that the expectation of c i,j is |D| if the negative sampling is assumed to follow unigram probability c j |D| , and the negative sampling data is k-times larger than the training data D, where The above matches 'shifted PMI' as described in  when we substitute c i,j for In addition, the word2vec implementation uses a smoothing factor α to reduce the selection of high-occurrence-frequency words during the negative sampling.