Since it is difficult to define the usefulness of the tuples without assuming a KB, we used relevance and concreteness as criteria to grade extracted tuples.
each worker was asked to choose one option from the three options: Not relevant or unclear (0), Relevant (1), Relevant and concrete (2).