We hypothesise that in general, the norm difference indicates the divergence between optimizers' steps and we observe that the larger the norm difference, the greater the change in the trajectory.
we propose to Algorithm 2: Norm-filtered Aggressive SWA algorithm Require: 1: e = Epoch number 2: m = Total epochs 3: i = Iteration number 4: n = Total iterations 5: α = Learning rate 6: O = Stochastic Gradient optimizer function 7: N s = List of previous iterations' norm differences e ← 0; while e < m do e ← e + 1 maintain a list that stores the norm differences of the previous iterations.