So motivated, after training the SWEM-max model on the Yahoo dataset, we selected five words with the largest values, among the entire vocabulary, for each word embedding dimension (these words are selected preferentially in the corresponding dimension, by the max operation).
sWEM exhibits comparable performance on the other two tasks, again with much less parameters and faster training.