That is, given a word w in a sentence, all the words corresponding to its immediate parent (-1), immediate children (+1), grandparent (-2) and grandchildren (+2) are extracted as features to constitute a context vector.
each feature in the vector is weighted by its point-wise mutual information (PMI) with the word w, defined as: where N(w, c) is the co-occurrence frequency of word w and its context word c, N(w) and N(c) are the occurrence frequencies of the words w and c respectively, N is the number of all word occurrences in the corpus.