We then provide empirical studies to validate design decisions.
we prove the necessity of both paragraph-level retrieval and sentencelevel retrieval for maintaining good performance, and further illustrate that a better semantic retrieval module not only is beneficial to achieving high recall and keeping high upper bound for downstream task, but also plays an important role in shaping the downstream data distribution and providing more relevant and high-quality data for downstream sub-module training and inference.