Previous work has shown that for many common NLP tasks, 7 Turkers' average score can match expert annotations (Snow et al., 2008).
we use 15 Turkers because we had no gold-standard data and because we were not sure how difficult the task is.