We tested the relation set with an initial inter-annotator agreement study (our latest interannotator agreement study results are presented in Section 6).
the mediocre results indicated that the categories and/or their definitions needed refinement.