Each annotation instance is assigned to two annotators randomly, so we can assume that the difficulty of the data assigned to each annotator is the same.
we calculate the average inter-annotator agreement of all pairs of annotators.