This has the effect of decreasing the variance, and thus the learning rate, for frequently updated features, preventing large and aggressive moves of weights for features that already have been seen often and thus can be considered relatively reliable.
2013) in which the optimization is performed in two steps: (t) ,Ẽ (t) ,C (t) ; w (t) ) (71) First, we perform updates without considering the regularization term in Equation 71.