We find that using adaptive learning rates for the optimization is efficient and leads the model to converge faster.
we use Adam (Kingma and Ba, 2015) with a learning rate of 0.0002 instead of stochastic gradient descent.