For a given system result S, gold standard G and its fuzzy version G * , we propose that precision be computed in the traditional way for the crisp version of the gold standard -P = |T P | -with the intuition that false positives proposed by the system (type I error) be weighted equally: if the system proposes an annotation, it should be correct, independently of the type of annotation.
a gold standard annotation not proposed by the system may be due to different design choices; we hence propose to use a fuzzy recall measure with respect to G * , namely R * = a∈S µ G * (a) a∈G µ G * (a) , thus applying different costs for missing annotations (type II errors) depending on the annotation in question.