Consider the following situation.
κ assessments for discourse and dialogue tagging are most often performed on the same portion of the data, which has been annotated by each of a small number of annotators (between two and four).