We use REINFORCE (Williams, 1992), one instance of the policy gradient methods as the optimization algorithm.
for each episode, the weights of the policy network are updated as follows: where v i = ∑ T t=i γ t−i r t is the culmulative future reward at time i and γ ∈ [0, 1] is a discounting factor of future rewards.