The reward R(Ŷ s ) is then defined as the difference between the sequence-level accuracy (A) of the sampled sequenceŶ s and the baseline sequenceŶ b .
the corresponding self-critical policy gradient loss is: minimizing the RL loss is equivalent to maximizing the likelihood of the sampledŶ s if it obtains a higher sequence-level accuracy than the baselineŶ b .