In order to improve generalization, we estimate the categorical parameters based on the counts from all instances, except the one whose gradient is being computed for the online update (leave-one-out).
we subtract the counts for a particular instance before computing the update (Equation 8) and add them back when the update has been executed.