We use softmax selection, which is widely applied in practice, to trade-off exploration vs. exploitation.
for each experiencereplay, we first generate 200 episodes of data (about 3K tuples in "Saving John" and 16K tuples in "Machine of Death") using the softmax selection rule in (2), where we set α = 0.2 for the first game and α = 1.0 for the second game.