ROUGE is widely used to automatically evaluate summarization systems.
we build upon this line of work and show that cosine similarity between the reference and summary embedding works well, and better than ROUGE on recent datasets, for comparing single document summarization systems.