Automatic Rule Refinement Tools automatically suggesting entire customization rules to a complex NERL program in the spirit of (Liu et al., 2010) can further reduce human effort in building NER annotators.
similarly, tools for explaining false positives in the spirit of (Huang et al., 2008), are also conceivable.