For a word type w 1 , the left-word context of w 1 , L(w 1 ), is defined as the N-dimensional vector whose n-th component is the number of bigrams, i.e., pairs of consecutive tokens (t i-1 , t i ) in the corpus, such that w(t i ) = w 1 and w(t i-1 ) = n. Similarly, we define the right-word context of w 1 , R(w 1 ), as the N-dimensional vector whose n-th component is the number of bigrams (t i , t i+1 ) such that w(t i ) = w 1 and w(t i+1 ) = n. We let L (resp.
for a probabilistic assignment P, we define G(P) to be the weighted average, over all w and all k, of ||L P (w) -µ L k || 2 + ||R P (w) -µ R k || 2 ; the weight used in this average is P wk f(w), just as in the computation of the Gaussian means.