In the spirit of PCA (Jolliffe, 2011), to keep the variance of A, we would like to have (AW ) c,: and (AW ) c ′ ,: distant from each other if c and c ′ are very different.
a pre-trained word embedding, which projects words with different meanings into different positions in space, is a natural choice for the projection matrix W and can help preserve much of the variance of A.