If we can identify those data, then a natural way to enhance the underlying classifier's performance would be to include them in the training data, and then retrain.
a human is required to obtain labels for the new data, but our goal is to reduce the human labeling effort as much as possible.