For instance, we observed that a total of 128 turkers have provided the flat annotations for Q700, when we had only asked for 10 annotations per query.
on average, a turker has annotated only 7.81% of the 700 queries.