In ridit scoring, ordinal labels are mapped to (0, 1) using the empirical cumulative distribution function of the ratings given by each annotator.
for the responses y (a) given by annotator a, ridit y (a) y Ridit scoring has the effect of reweighting the importance of a scale label based on the frequency with which it is used.