Compared with the previous two approaches, a language model, trained on an enormous amount of text, can naturally capture coherence among both words and utterances.
a good evaluation metric should not only measure the quality of gen-eration, but also the diversity of generation, which is especially important for open-ended tasks like dialogue or story generation (Hashimoto et al., 2019).