Denoting the predicted restated query byz, simplifying E (x,y,z)∼D as E, the goal of the RL training is to maximize following objective: where Z is the space of all restated query candidates and r represents the reward defined by comparingz and the annotation z.
the overall candidate space Q×Z is vast, making it impossible to exactly maximize L rl .