However, we also hoped that a third technique, active learning (Cohn et al., 1996;McCallum and Nigam, 1998), would be particularly effective when used in conjunction with hierarchical word clusters.
active learning attempts to select examples for annotation by estimating the system's certainty about the answer, requesting a human judgment only for those cases where it is most uncertain.