Standard results from the stochastic approximation literature state that ∞ k=0 η k = ∞ and ∞ k=0 η 2 k < ∞ are sufficient to guarantee convergence to a local optimum.
if we take η k = (k + 2) −α , then any 0.5 < α ≤ 1 is valid.