When successful, crowdsourcing enables gathering annotations at scale; however, its performance is still limited by (1) the difficulty of expressing the annotation task as a simply-understood task suitable for the layman, (2) the cost of collecting many annotations, and (3) the tediousness of the task, which can fail to attract workers.
several groups have proposed an alternate annotation method using games: an annotation task is converted into a game which, as a result of game play, produces annotations (Pe-Than et al., 2012;.