True label discovery methods have been developed to resolve conflicts among multi-source information under the assumption of source consistency (Li et al., 2016;Zhi et al., 2015).
in the spammer-hammer model (Karger et al., 2011), each source could either be a spammer, which annotates instances randomly; or a hammer, which annotates instances precisely.