A very few works try structured weight pruning, e.g., Wang et al. (2019) proposes a structured pruning approach based on low-rank factorization and augmented Lagrangian L0 norm regularization.
there also exist works that prune a coherent set of sub-modules in the Transformer model.