Further analysis reveals that about 12.6% of dev questions and 15.7% test ones are considered unanswerable by crowd workers, which is a byproduct of the information-asymmetric setting adopted when the data was collected.
many reference questions could be considered uninformative by our definition, since they might cause the QA model to abstain from answering.