In word similarity, spectral methods generally excel, with CCA consistently performing the best.
it has been found by many (including ourselves) that setting β = 1 yields substantially worse representations than setting β ∈ {0, 0.5} (Levy et al., 2015).