They can capture semantic and syntactic properties of words as linear substructures, allowing relationships to be expressed as geometric translations (Mikolov et al., 2013b).
it is surprising that linear maps also outperform geometric translations, given that they are ultimately learned using translation vectors.