This paper explores recent refinements to the nugget-based evaluation methodology developed by NIST.
we examine the recent so-called "pyramid extension" that incorporates relevance judgments from multiple assessors to improve evaluation stability (Lin and Demner-Fushman, 2006).