In other words, for the SGD update rule F SGD , the fixed learning rate γ max is derived from the ADF update rule.
according to Lemma 1, the SGD update rule F SGD is a contraction mapping in Euclidean space with Lipschitz continuity degree 1 − γ max /σ 2 , given the condition that γ max ≤ (||x 2 Combining Equations (6) and (7), it goes to |F(w w w t+1 ) − F(w w w t )| ≤ (1 − γ max /σ 2 )|w w w t+1 − w w w t | according to the definition of dynamic contraction mapping, the aDF update rule is a dynamic contraction mapping in Euclidean space with Lipschitz continuity degree 1 − γ max /σ 2 .