To ensure that the policies do not converge by chance, we run the training and test sessions 20 times each and we report averages.
clearly having a constant exploration rate in all epochs is problematic.