To optimize the inner product between c and w p(w|c)w, we essentially align the direction of context vector c with the expected word vector under context c, w p(w|c)w = E w∼p(w|c) w. In other words, given a word embedding matrix W ∈ R V ×d , our approach aligns c with the corresponding row (AW ) c,: in AW .
the objective can be viewed as conducting multivariate regression to approximate (AW ) c,: given the context.