A major difference between what the state-of-theart models do and what our model does is that our model distributes a probability mass of one across the vocabulary, while the thematic fit models have no such overall constraint; they will assign a high number to all words that are similar to the prototypical vector, without having to distribute probability mass.
this implies that two synonymous fillers, one of which is a frequent word like fire, and the other of which is an infrequent word, e.g., blaze, will get similar ratings by the distributional similarity models, but quite different ratings by the neural network model, as the more frequent word will have higher probability.