Second, since multiple questions are valid but only one reference exists in the dataset, these traditional metrics fail to appropriately score question paraphrases and novel generation (shown in Table 2).
we introduce a QA-based evaluation method that directly measures the QG model's ability to mimic human annotators in generating QA training data, because ideally, we hope that the QG model can act like a human to ask questions.