Thus we calculated informativeness scores based on the common information between the output of the MSC system and the references using ROUGE (Lin, 2004).
we used the f-measure metrics ROUGE-1, ROUGE-2 and ROUGE-SU4.