Due to lack of labeled data, the scoring mechanism cannot distinguish between noisy and non-noisy extractors.
an extractor is judged by its ability to extract more positive and less negative extractions.