Traditionally, labels are collected from multiple annotators/experts who are assumed to provide reliable labels.
in reality, these experts may have varying levels of expertise depending on the domains, and thus may disagree on labelling in certain cases (Aroyo and Welty, 2013).