This idea is similar to the mini-batch learning, which has been used in large-scale SVM problem (Nelakanti et al., 2013) and deep learning models (Le et al., 2011).
in iteration t, the algorithm randomly chooses a subset of training samples D t to train the model.