As we show in Section 2, the predominant gold standards for semantic evaluation in NLP do not measure the ability of models to reflect similarity.
in both WS-353 and MEN, pairs of words with associated meaning, such as coffee and cup (rating = 6.810), telephone and communication (7.510), or movie and theater (7.710), receive a high rating regardless of whether or not their constituents are similar.