In the dialog domain, single-reference BLEU score (Papineni et al., 2001) is sometimes used to measure response quality, but it has been shown to have little correlation with human-judged quality (Liu et al., 2016).
most works in dialog systems use human evaluation as the ultimate measure of quality (Li et al., 2016a;Sedoc et al., 2018) 8 Conclusion In this work, we perform an analysis of posttraining decoding strategies that attempt to promote diversity in conditional language models.