GLSA begins with a matrix of pair-wise term similarities S, computes its eigenvectors U and uses the first k of them to represent terms and documents, for details see (Matveeva et al., 2005).
the justification for this approach is the theorem by Eckart and Young (Golub and Reinsch, 1971) stating that inner product similarities between the term vectors based on the eigenvectors of S represent the best elementwise approximation to the entries in S. the inner product similarity in the GLSA space preserves the semantic similarities in S. Since our representation will try to preserve semantic similarities in S it is important to have a matrix of similarities which is linguistically motivated.