Even fine-tuning the pre-trained model with taskspecific dataset may take several hours to finish one epoch.
reducing computational costs for such models is crucial for their application in practice, where computational resources are limited.