In fact, we found that larger ηs do not delay most training instances in the first few iterations.
once the network obtains a reasonably high performance, schedulers start delaying instances for longer durations.