The underlying intuition known as distributional hypothesis (Harris, 1954) can be summarized as: "a word is characterized by the company it keeps" (Firth, 1957).
there is a lack of theory to justify why it works, even for a simple task such as word classification or clustering.