Previous research has used very large batches to improve training convergence while requiring fewer model updates (Smith et al., 2017;Neishi et al., 2017).
with such large batches the model size may exceed available GPU memory.