As the goal of our experiments is to demonstrate the ability of our approach to reduce the number of parameters, we only consider rational baselines: the same rational RNNs trained without group lasso.
our heavily regularized models perform substantially better than the unigram baselines, gaining between 1-2% absolute improvements in four out of five cases.