Our proposed optimization methods introduce a trust-region-type regularization (Conn et al., 2000) at each iteration, and then update the model only within a small neighborhood of the previous iterate.
they can effectively prevent aggressive updating and stabilize the finetuning process.