4 Expected agreement is then the probability of c 1 and c 2 agreeing on any category, that is, the sum of the product over all categories: The difference between S, π, and κ lies in the assumptions leading to the calculation of P(k|c i ), the chance that coder c i will assign an arbitrary item to category k (Zwick 1988;Hsu and Field 2003).
this coefficient is based on the assumption that if coders were operating by chance alone, we would get a uniform distribution: for any two coders c m , c n and any two categories k j , k l , P(k j |c m ) = P(k l |c n ).