Evaluating approaches to identifying diachronic semantic change is difficult; indeed, most previous studies have relied on rather small datasets (e.g., Sagi et al., 2009;Cook and Stevenson, 2010) or human judges' intuitions about changes in meaning, which might not accurately reflect changes in meaning observed in corpora (e.g., Gulordava and Baroni, 2011).
taking inspiration from evaluation approaches in word sense disambiguation that make use of artificiallyambiguous words (e.g., Schütze, 1992;Gale et al., 1992), we propose evaluation methods that use synthetic examples of semantic change.