The language model proposed by Huang et al. (2012) exploits both local and global context that are acquired through a joint training objective.
word representations are computed while learning to discriminate the next word, given a local context composed of a short sequence of words, and a global context composed of the whole document where the word sequence occurs.