The difference is that in EPCA, the representations wj are learned jointly with C. 5 In the general form of EPCA, that column is passed through some "inverse link" function to obtain the expected feature values under the distribution, which in turn determines EPCA allows us to suppose that each x j was drawn from a multinomial-a more appropriate family for drawing a count vector.
our observation is that skip-gram is precisely multinomial EPCA with the canonical link function (Mohamed, 2011), which generates x j from a multinomial with log-linear parameterization.skipgram chooses embeddings C, W to maximize This is the log-likelihood (plus a constant) if we assume that for each word j, the context vector x j was drawn from a multinomial with natural parameter vector C w j and count parameter N j = i X ij .