Background While sequence models are usually trained using XENT, they are typically evaluated at test time using discrete NLP metrics such as BLEU (Papineni et al., 2002), ROUGE (Lin, 2004), METEOR (Banerjee and Lavie, 2005).
they suffer from both the exposure bias and non-differentiable task metric issues.