Following recent formulations (Rajpurkar et al., 2016), we further assume that a i appears as a substring for some document in the set D i .
the oracle score is the upper bound on the exact match accuracy.