Pre-training on a source RC dataset and transferring to the target improves performance by 21% on average for DOCQA (improving on 8 out of 11 datasets), and by 7% on average for BERTQA (improving on 10 out of 11 datasets).
pre-training on a related RC dataset helps even given representations from a model like BERTQA.