We were able to obtain ratings on the entire Ziff-Davis test set as it has only 32 instances; this was not possible for Broadcast news as the test section consists of 410 instances.
we chose F1 (as opposed to accuracy or edit distancebased measures) as Clarke and Lapata (2006b) show that it correlates reliably with human judgements.