It should be borne in mind that it would not be difficult to construct a scenario in which the ramifications for a MUC-like task would have been far more severe--with a different task specification, template structure, set of slot definitions, or scoring scheme, for instance--to the extent that the integrity of such an evaluation could be compromised.
these results serve to highlight a potential pitfall to be avoided in future evaluations of IE and other high-level language processing tasks, which, like the MUCs, may be the principle driving forces behind technology development for extended periods of time.