On the other hand, the objective J θ of the reward function is to distinguish between humanannotated stories and machine-generated stories.
hence it is trying to minimize the KL-divergence with the empirical distribution p e and maximize the KL-divergence with the approximated policy distribution π β : Since h(π β ) and h(p e ) are irrelevant to θ, we denote them as constant C. the objective J θ can be further derived as here we propose to use stochastic gradient descent to optimize these two models alternately.