In the TAC 2014 summarization track, ROUGE was suggested as the evaluation metric for summarization and no human assessment was provided for the topics.
to study the effectiveness of the evaluation metrics, we use the semi-manual Pyramid evaluation framework (Nenkova and Passonneau, 2004;.