With this aim, we extended the above metrics at the document level (i.e., questions and answers are generated from the source article text rather than from the reference summary), dispensing of the need for human-generated reference summaries.
in line with the APES approach described above, we propose two unsupervised QA-based metrics, to which we refer to as QA f score (unsup) and QA conf (unsup).