4 Part-of-speech tags are the ones assigned by our cost-sensitive (CM) POS model trained on Twitter data, the concatenation of Gimpel and 70% Ritter training data.
the measure used in this paper is less affected by these factors, but manages to give us a good understanding of the agreement.