In addition, since ADADELTA computes the moving average of updates, and ADAM needs to compute the bias-corrected gradient estimate, they require more intricate implementations.
since the update progresses slowly, online LDA needs more iterations to break the equilibrium.