We train a simple one-layer convolutional neural network (CNN) on top of the word embeddings, which has shown to perform well on text classification tasks regardless of training data size (Kim, 2014).
convolutional filters are applied to windows of word embeddings, with a max-over-time pooling on top of them.