It is possible that by focusing more on the training of annotator pairs, particularly on joint training, agreement might improve.
that would also result in a bias, which is probably not preferable to actual perception.