However, using pretrained word2vec embeddings, we also achieve 3% advantage.
it seems that context-sensitive signals plus the generalizing power of embeddings contain all the context-free signals already.