Vlachos (2008) also studied a stopping criterion of active learning based on the estimate of the classifier's confidence, in which a separate and large dataset is prepared in advance to estimate the classifier's confidence.
there is a risk to be misleading because how many examples are required for this pregiven separate dataset is an open question in real-world applications, and it can not guarantee that the classifier shows a rise-peak-drop confidence pattern during active learning process.