As shown in Figure 1, the fine-tuning path from the start point to the end point on the loss landscape is more smooth than training from scratch.
the training loss of fine-tuning BERT tends to monotonously decrease along the optimization direction, which eases optimization and accelerates training convergence.