To fully harness the power of fine-tuning in a more principled manner, we propose a new learning framework for robust and efficient fine-tuning on the pre-trained language models through regularized optimization techniques.
our framework consists of two important ingredients for preventing overfitting: (I) To effectively control the extremely high complexity of the model, we propose a Smoothnessinducing Adversarial Regularization technique.