2 A useful measure of quality is reproducibilityif repeated independent manual annotations from multiple respondents result in similar sentiment scores, then one can be confident that the scores capture the true sentiment intensities.
we set up an experiment that compares BWS and RS in terms of how similar the results are on repeated independent annotations.