(2018) showed how the reported high performance of multi-prototype techniques in this dataset was not due to an accurate sense representation, but rather to a subsampling effect, which had not been controlled for in similarity datasets.
a context-insensitive word embedding model would perform no better than a random baseline on our dataset.