For all model adjustment experiments, we use the architecture of our suggestion models described in Section 4.1.
in this section, we examine the effects of annotation suggestions in detail, considering interannotator agreement, intra-annotator consistency, annotation bias and speed, as well as usefulness of suggestions and the impact of universal versus personalised suggestion models.