The analyses expose a number of challenges and several surprising results.
while ROUGE has very strong correlation with responsiveness for both human and system summaries, there is a significant gap in responsiveness between humans and systems which is not accounted for by the ROUGE metrics.