Surprisingly, on topic prediction tasks, our SWEM model exhibits stronger performances, relative to both LSTM and CNN compositional architectures, this by leveraging both the average and max-pooling features from word embeddings.
our SWEM-concat model even outperforms a 29-layer deep CNN model (Conneau et al., 2016), when predicting topics.