Testing Automatic Response Models In testing a retrieval-based system, although we can simply use the original responses associated with the query post as positive and treat all the others as negative, this strategy suffers from the problem of spurious negative examples.
with a reasonably good model, the retrieved responses are often good even if they are not the original ones, which brings significant bias to the evaluation.