3 In this example, we experiment with increasing dimensions in the recurrent layer while maintaining the number of trainable parameters, whereas in Section 4.2 we increase sparseness while maintaining dimensions.
each LSTM layer is made sparse in such a way that the hidden dimension 1150 is increased by a factor 1.5 (chosen ad hoc) to 1725, but the embedding dimensions and total number of parameters remain the same (within error margins from rounding to integer dimensions for the dense blocks).