We also tried computing the diagonal ofF on held-out data, as 5 github.com/thompsonb/sockeye_ewc there is some evidence that estimating Fisher on held out data reduces overfitting in natural gradient descent (Pascanu and Bengio, 2013).
we again found no meaningful differences.