Since our model is a non-linear approximation of the underlying action-value function of the game, we learn model parameters by applying non-linear regression to the observed final utilities from the simulated roll-outs.
we adjust the parameters by stochastic gradient descent, to minimize the mean-squared error between the actionvalue Q(s, a) and the final utility R(s τ ) for each observed game state s and action a.