The standard methodology for analyzing crowdsourced data in NLP is based on majority voting (selecting the label chosen by the majority of coders) and inter-annotator coefficients of agreement, such as Cohen's κ (Artstein and Poesio, 2008).
aggregation by majority vote implicitly assumes equal expertise among the annotators.