This results in difficulties for current reference-based evaluation, especially of fluency, causing wordoverlap metrics like ROUGE (Lin and Och, 2004) to correlate only weakly with human judgments (Toutanova et al., 2016).
fluency evaluation of NLG is often done manually, which is costly and time-consuming.