The latent assumption of the word generator is that each word is independent and the score is irrelevant to each other.
the scores of a word and its synonyms may be of great difference, which means the word generator learns the word itself rather than the relationship between words.