Our aim is to analyze the effectiveness of the evaluation metrics, not the summarization approaches.
we consider the correlations of the automatic evaluation metrics with the manual Pyramid scores to evaluate their effectiveness; the metrics that show higher correlations with manual judgments are more effective.