The first improvement featured larger word embedding vectors (of the size 300 instead of 50), which gave 6647 features for each training example.
despite much richer embeddings, we did not observe any significant improvements in the evaluation metrics.