This embedding was shown to improve results on downstream tasks (Baroni et al., 2014;Bullinaria and Levy, 2012;Turney and Pantel, 2010).
the top 300 elements of the diagonal matrix of singular values Σ, denoted Σ d , were retained to represent a new, dense embedding of the word vectors, using the truncated left hand orthonormal matrix U d : These representations were subsequently aligned with the orthogonal Procrustes method following (Hamilton et al., 2016).