Our results show a solid 11-26% reduction in perplexity with respect to the feed-forward model across three corpora of different sizes and genres when the model uses MLP Convolution and combines kernels of different window sizes.
even without these additions we show CNNs to effectively learn language patterns that allow it to significantly decrease the model perplexity.