For solving this, many dense annotation schemata are proposed to force annotators to annotate more or even complete graph pairs.
dense annotation is time-consuming and unstable human judgments on "salient" pairs are not improved at all.