Instead, we propose slanted triangular learning rates (STLR), which first linearly increases the learning rate and then linearly decays it according to the following update schedule, which can be seen in Figure 2: where T is the number of training iterations 4 , cut f rac is the fraction of iterations we increase 3 An unrelated method of the same name exists for deep Boltzmann machines (Salakhutdinov and Hinton, 2009).
4 the number of epochs times the number of updates per epoch.