Training in machine learning often uses starting big which is to use up all training data at the same time.
elman (1993) suggests that in some cases, learning should start by training simple models on small data and then gradually increase the model complexity and add more difficult data.