Moreover, the RA measure is extremely sensitive to the inconsistency of results, as the presence of one incongruous annotation out of 5 is enough to lower the outcome.
we used a second measure to take into consideration the difficulty of having 5 people agree.