Moreover, models are usually agnostic to the noises or artifacts of the training data, such as reference divergence, making them vulnerable to hallucinations (Kryscinski et al., 2019a;Wiseman et al., 2017;Dhingra et al., 2019).
models can generate texts that are not consistent with the input, yet would likely have reasonable model log-likelihood.