Recent work by Dima (2016) and Vered and Waterson (2018) suggest that the gains achieved in noun-noun compound interpretation using word embeddings and somewhat similar neural classification models are in fact a by-product of lexical memorization (Levy et al., 2015); in other words, the classification models learn that a specific set of nouns is a strong indicator of a specific relation.
in order to gauge the role of lexical memorization in our models also, we quantify the number of unseen compounds that the STL, TL and MTL models predict correctly.