To better understand the limits and potential of our proposed method, we further study about the QA IOU F1 Token F1  6.3 13.9 Bert-To-Bert 2019  ability and comprehensiveness of our model.
for the learning rate, we found that when using 1e - 5 without warm-up, the valid loss of SQuAD is 10% higher than using 5e - 6.