Inducing sparseness while training neural networks has been shown to yield models with a lower memory footprint but similar effectiveness to dense models.
sparseness is typically induced starting from a dense model, and thus this advantage does not hold during training.