We observe that the original SGD reaches the desired minima, however, it almost reaches the saddle point and does a course correction and reaches minima.
we observe that SGD with ASWA is very conservative, it repeatedly restarts and reaches the minima without reaching the saddle point.