However in contrast to the support set in SVMs, an example is added to the active set every time the online algorithm makes a prediction mistake or when its confidence in a prediction is inadequately low.
the active set can increase in size significantly and thus the total computational cost becomes proportional to the square of the number of training examples.