Finally, all three translators met and performed a final check, with specific focus on the following two cases: (1) duplicate pairs in the dataset, and (2) pairs with repeated words.
due to the lack of standard multilingual benchmarks, word similarity systems had in the main been limited to the English language (Mihalcea and Moldovan, 1999;Agirre and Lopez, 2003;Agirre and de Lacalle, 2004;Strube and Ponzetto, 2006;Gabrilovich and Markovitch, 2007;Mihalcea, 2007;Pilehvar et al., 2013;, up until the recent creation of datasets built by translating the English RG-65 dataset (Rubenstein and Goodenough, 1965) into French (Joubarne and Inkpen, 2011), German (Gurevych, 2005), and Portuguese (Granada et al., 2014).