On the MC500 test set, we again find that model 3, with a score of 63.75%, provides a gain of 3.5% over SWD and is comparable to the performance of RTE+SWD (63.33%) The importance of utilizing multiple relevant sentences to score answers is evident from the higher scores of models 2 and 3 on multi type questions in both test sets.
model 1, which retrieves only a single relevant sentence for each question, achieves the best scores on the single type questions up to 83.25% on MC160 test.