Ranzato et al. (2016) propose MIXER algorithm, which is the first application of REINFORCE algorithm (Williams, 1992) in training sequence to sequence model.
an additional model, which is used to predict expected reward, is required in MIXER.