For efficiency reasons, in practice, we only consider a small subset of all context words j for which x ij = 0, which is similar in spirit to the use of negative sampling in Skip-gram (Mikolov et al., 2013b).
the set J i contains each j such that x ij > 0 as well as M uniformly 1 sampled context words j for which x ij = 0, where we choose M = 2 • |{j : Second, following (Jameel and Schockaert, 2016), the weighting function f (x ij ) has been replaced by 1 j is the residual variance of the regression problem for context word j, estimated follows: Since we need the word vectors to estimate this residual variance, we reestimate σ 2 j after every five iterations of the SGD optimization.