If online reinforcement learning is used, we will be unable to evaluate the generated policy after each update, and hence will not know which policy to keep for the final evaluation.
note that, GPTD can also admit recursive (online) computations, but here we focus ourselves on the batch version.