Actor-critic (Sutton and Barto, 1998) methods are often adopted, which involves training an additional value network to predict the expected reward.
(Rennie et al., 2017) designed a self-critical method that utilizes the output of its own test-time inference algorithm as the baseline to normalize the rewards, which leads to further performance gains.