If the kernel-trick (Taylor and Cristianini, 2004) is applied to online margin-based learning, a subset of the observed examples, called the active set, needs to be stored.
one way to deal with this is to filter out low-confidence features, but it is difficult to decide which features are important in online learning.