In other words, the training loss of fine-tuning BERT tends to monotonously decrease along the optimization direction, which eases optimization and accelerates training convergence.
the path from random initial point to the end point is more rough, which requires a more carefully tweaked optimizer to obtain reasonable performance.