A general recommendation would be to adjust the minimum level of agreement depending on the annotation purpose-as is already the practice in the computational linguistics community as indicated by our data.
inter-annotator agreement decreased with a rising number of categories in the coding scheme.