This approach, however, requires a set of high-quality labeled data (i.e., the Gold Standard) for providing the instruction and feedback to the crowd workers.
acquiring such data requires a considerable amount of human effort.