Donner and Eliasziw (1987) propose a more general form of significance test for arbitrary levels of agreement.
krippendorff (2004a, Section 11.4.2) states that the distribution of α is unknown, so confidence intervals must be obtained by bootstrapping; a software package for doing this is described in Hayes and krippendorff (2007).