The Random Indexing approach is motivated by the Johnson-Lindenstrauss Lemma (Johnson and Lindenstrauss, 1984), which states that if we project points into a randomly selected subspace of sufficiently high dimensionality, the distances between the points are approximately preserved.
if we collect the random index vectors into a random matrix R of order c × k, whose row R i is the k-dimensional index vector for context i, we find that the following relation holds: That is, the Random Indexing context matrix G contains the same information as we get by multiplying the standard cooccurrence matrix F with the random matrix R, where RR T approximates the identity matrix.