In addition, we would like to apply the proposed methods for other pretrained models.
the training loss of fine-tuning BERT tends to monotonously decrease along the optimization direction, which eases optimization and accelerates training convergence.