This is similar to the the Co-  gALex shared task , where the first part of the task is to eliminate completely unrelated pairs, before predicting relations on the remaining pairs.
filtering out unrelated pairs is an easier task than filtering pairs in the Other category.