Since our model is a non-linear approximation of the underlying action-value function of the game, we learn model parameters by applying non-linear regression to the observed final utilities from the simulated roll-outs.
the success of Monte-Carlo search is based on its ability to make a fast, local estimate of the ac-procedure PlayGame () Initialize game state to fixed starting state Compute average observed utility for each action Sample action from action-value function in -greedy fashion: Execute selected action in game: if game is won or lost break end Update parameters w of Q(s, a) Return action and observed utility: Algorithm 1: the general Monte-Carlo algorithm.