Given this ranking task, our evaluation is inspired by the TREC competitions and work in information retrieval (Manning et al., 2008).
we treat each relation as query and receive the top 1000 (run depth) entity pairs from each system.