However, now we are interested in both outcomes of the classification, while earlier we were only interested in knowing how well we were finding relations (and not in how well we were finding "non-relations").
accuracy is the relevant metric (  Table 5: System for the task of relation classification.