Interestingly, the impressive results of these models are achieved despite the shallow linguistic information most of them consider, which is limited to the tendency of words to co-occur together in a pre-specified context window.
very little information is encoded about the syntactic and semantic relations between the participating words, and, instead, a bag-of-words approach is taken.