Prior work has shown that commonly used metrics for overall quality in other generation tasks such as BLEU (Papineni et al., 2002), ROUGE (Lin, 2004), ME-TEOR (Banerjee and Lavie, 2005) and perplexity have poor correlations with human judgment (Liu et al., 2016;Tao et al., 2018) 4 or are modeldependent (Liu et al., 2016).
we adopt several metrics that evaluate multiple aspects of responses, and also conduct human evaluation for each result we present.