In the SimLex-999 dataset which focuses on the word similarity, the cognition (lexical relation) and the sentiment modules turned out to be important.
in the WordSim-353 dataset which focuses on the word relatedness, both linear context and syntactic context are turned out to be critical.