After observing the TriviaQA dataset, we find that in this dataset only one or two retrieved paragraphs actually contain the correct answer.
simply using all retrieved paragraphs equally to extract answer may bring in much noise.