For reasons to be discussed in Section 5, we instead prefer a soft-assignment strategy, inspired from the EM algorithm for the estimation of a mixture of Gaussians.
rather than the hard assignment A, we use a soft-assignment matrix P. P wk , interpreted as the probability of assigning word w to cluster k, is, essentially, proportional to exp{-d wk 2 /2σ 2 }, where d wk is the distance between the latent descriptor for w and the centroid, i.e., Gaussian mean, for k. Unlike the Gaussian-mixture model however, we use the same mixture coefficient and the same Gaussian width for all k. Further, we let the Gaussian width σdecrease gradually during the iterative process.