When using ADA-GRAD (Duchi et al., 2011) with online LDA, the update rule for the each learning rate is where ρ 0 is a constant, and a very small guarantees that the learning rates are non-zero.
it may seem reasonable to apply ADA-GRAD to optimize online topic models.