Since the reward estimator of GDPL evaluates state-action pairs, it can always guide the dialog policy during the conversation thus leading the dialog policy to a successful strategy, which also indirectly demonstrates that the reward estimator has learned a reasonable reward at each dialog turn.
the loss functions for the dialog policy and the reward estimator become respectively as follows: where T is the number of dialog turns.