We now consider an on-line algorithm, stochastic gradient descent (SGD) (Robbins and Monro 1951), which updates the parameters after computing the candidate set for each example.
we iteratively scan through the training examples in a random order.