Open-ended generation -the task of generating text that forms a natural continuation from the input text -requires the model to hallucinate text; hence the focus has been to ensure that the model learns to generate text that is more human-like (i.e., less repetitive or dull with more content-related words) (Holtzman et al., 2020;Welleck et al., 2020;See et al., 2019).
tasks such as document summarization (Nenkova and McKeown, 2011;See et al., 2017;Paulus et al., 2018) and data-to-text generation (Lebret et al., 2016;Wiseman et al., 2017) which are not open-ended, require models to be factual and/or faithful to the source text.