However, since the boundaries of the spans annotated in the MPQA corpus are not strictly defined in the annotation guidelines (Wiebe et al., 2005), measuring precision and recall using exact boundary scoring will result in figures that are too low to be indicative of the usefulness of the system.
we used the function ρ(y,ŷ) = 1 − F (y,ŷ), where F is the soft F-measure described in Section 4.1.