We show that ROUGE F1 is unfortunately sensitive to summary output length, in general favoring models that produce longer summaries.
we argue that controlling the output length should be an integral part of the summarization task and that a fair system comparison can only be conducted between summaries in the same length bracket.