For the sake of these experiments we chose to stop after an hour of an annotator time (the initial lexicon expansion bootstrap and annotating/adjudicating 1,100 sentences).
the human annotator using RASCAL gets a fairly good sense of what kind of annotations are being spotted and what is being missed.