We conduct experiments with BERT on 5 GLUE benchmark tasks to demonstrate that SNIP achieves effective pruning results while maintaining comparable performance.
we improve the performance over the state-of-the-art by 0.5 to 1.0% on average at 50% compression ratio.