Second, loss landscapes of fine-tuning partially explain the good generalization capability of BERT.
pre-training obtains more flat and wider optima, which indicates the pre-trained model tends to generalize better on unseen data (Chaudhari et al., 2017;Li et al., 2018;Izmailov et al., 2018).