Unlike ACE, this allows evaluators to measure performance without exhaustively annotating documents, allows for balance between rare and common relations, and implicitly measures coreference without requiring explicit annotation of answer keys for coreference.
because the evaluation only measures performance on the set of queries, many relation instances will be unscored.