Inter-annotator agreement was comparable to related semantic annotation tasks that employ crowdsourcing.
we annotated a large set of 9856 propositions regarding well-formedness and used the wellformed propositions to generate 12403 proposition pairs for the annotation of context-sensitive semantic verb relations.