To evaluate the multilanguage-comparability of subjectivity analysis systems, we measure 1) how consistently the system assigns subjectivity labels and 2) how closely numeric scores for systems' confidences correlate with regard to parallel texts in different languages.
we use Cohen's kappa coefficient for the first and Pearson's correlation coefficient for the latter.