Since the learning problem is nonconvex, different initializations of the parameters yield different solutions to the learning problem.
for any given experiment, we ran multiple random restarts for every setting of our hyperparameters and picked the model that performed best using the held-out tune set.