We hypothesized that the training data labels may have been highly sensitive to the subjectivity of the reviewers.
we retrained the learner with only those observations made by reviewers A and B (of the five total reviewers) and again compared the classification results with the gold standard.