The value is set as 1 at the start and gradually decreases to 0.1 after 3000 annealing steps.
we adopt Q-learning (Sutton and Barto, 1998) to optimize the neural agent, which uses a function Q(z, a) to represent Q-values and the recursive Bellman equation to perform Q-value iteration, when observing a new sample (z, a, z , r).