We collect a new set of distractor paragraphs to evaluate if the models are robust to the change in distractors.
6 we follow the same strategy as the original approach  using TF-IDF similarity between the question and the paragraph, but with no overlapping distractor paragraph with the original distractor paragraphs.