While we justified our choice of ROUGE-1 recall and ROUGE-2 recall as optimization objectives by their strong correlation with human evaluation methods, ROUGE-N has also various drawbacks.
it does not take into account the overall discourse coherence of a system summary (see the supplemental material for examples of summaries generated by our framework).