We use Spearmans' correlation coefficient as a scoring function.
for each sentence we create two vectors comprising counts of how many times each token was annotated by crowd and expert workers, respectively, and calculate the correlation between these.