These attractive properties were not studied in A2C's original paper (Mnih et al., 2016).
in any realistic scenario, human feedback will vary from its average, and so the reward that our algorithm receives will be a perturbed variant of sentence-BLEU.