Raters label pairs in batches of 20, meaning we have a minimum of 20 ratings per rater.
whether or not humans' judgments can be summarized by a single aggregate label or value might be a moot question, since state-of-the-art models do not, in practice, predict a single value but rather a distribution over values.