It is not straightforward to compare our classification result to the outcome of other annotation efforts, not only because there are only few such efforts, but also because annotation evaluation scenarios vary considerably.
qasemiZadeh and Schumann (2016) provide a detailed analysis of their manual annotation of the same data set.