We suspect this is also due to the insufficiencies of standard TF-IDF retrieval for multi-hop questions.
table 5 shows that single-paragraph BERt achieves 53.12 F1 even when using 500 distractors (rather than eight), indicating that 500 distractors are still insufficient.