In our preliminary experiments, we observed that SGD optimizer is more stable and less likely to quickly overfit the training set than other optimizers with adaptive learning rates (Reddi et al., 2018; Kingma and Ba, 2015).
we use SGD with the learning rate of 0.005 and a batch size of 128.