Even when brought to scale with the NLL loss, it shows much larger fluctuations as the training progresses.
it shows major increases at the re-starts of the optimizer for the simulated annealing that are not compensated for by the rest of the training.