This section takes a closer look at the setup of the definition question evaluation at TREC 2003.
we examine three issues: the scoring metric, error inherent in the evaluation process, and variations in judgments.