Note, methods using IR (e.g., RoBERTa+KE) must retrieve from Wikipedia during finetuning and inference, which increases the computational overhead significantly.
methods based on additional self-supervised pre-training are more efficient, but often achieve sub-optimal performance since they lack explicitly retrieved contexts.