The annotations for an item from multiple respondents are usually averaged to obtain a real-valued score for that item.
for an N -item set, if each item is to be annotated by five respondents, then the number of annotations required is 5N .