The favorable performance of the pruned and retrained models raises the question: can we get a shortcut to this performance by starting with sparse models?
rather than train, prune, and retrain, what if we simply prune then train?