In practice, this method produces significant speedups at both training and inference time (by selecting a small subset of columns and performing matrix multiplications given much smaller matrices).
it is reported to achieve lower performance compared to unstructured pruning (Yao et al., 2019) due to more restrictive sparse patterns