The basic premise behind our evaluation technique can be described as follows: The distributional spread of answers in a simple, three-way decision task should directly reflect the relative distance of words on an SO (semantic orientation) scale.
we can validate our dictionaries without forcing Turkers to use our 11-point scale (including zero), making their task significantly less onerous as well as less subject to individual bias.