Evaluation-The methods we concern are evaluated by the metrics of macro-average Precision (Mac-P), Recall (Mac-R) and F-score (Mac-F).
for a particular type t, the positive examples are defined as the event pairs that hold the relation t. for t, the precision score is calculated as the ratio of the positive examples in all the output examples of type t. The recall is defined as the ratio of the output positive examples in all the ground-truth ones.