For future work, we are investigating alternatives to MACE, which was designed for categorical annotations rather than the sequence labelling of our task.
we are looking into the Bayesian method of Simpson and Gurevych (2018), which takes advantage of the sequential dependencies between BIO tags, and works more robustly with noisy, subjective data such as ours.