Similarly to the trail study, the decision of whether or not an utterance includes the discourse function of a certain MD tag is made based on the overall agreement between annotators; this is because the primary objective of this study is to detect if an utterance contains an instance of metadiscursive acts.
agreement between annotators is considered to exist if the intersection (in terms of number of words) between their annotations is not void.