In such situations, which are common in information retrieval and entity recognition, metrics like precision and recall are typically used to describe performance.
precision and recall fail to describe the differences between sets of objects selected by different decision strategies, instead just describing the proportional amount of correct and incorrect objects selected.