In section 4 we examine the sources of disagreement among the annotators and in 5 we summarize the recommendations for reliable annotation.
(2009) raters found more than one valid construction for more than 18% of noun phrases.