This amount is large enough to ameliorate the flaws of BERT that has almost no questions on the left side and no textual span predictions based on both the question and the document on the right side.
a small amount of finetuning examples is not sufficient to turn BERT to be more task-aware, as shown in Sec.