We employ the REINFORCE algorithm (Williams, 1992) and policy gradient methods (Sutton et al., 2000) to optimize parameters of the agent network, where the key step is to rewrite the gradient formulation and then apply the back-propagation algorithm (Rumelhart et al., 1986) to update network parameters.
we define our objective as: where x denotes the input representation of the instance s. By taking the derivative of J(Θ) with respect to Θ, we can obtain the gradient Besides, we utilize the -greedy trick to balance exploration and exploitation.