The <unk> vector is part of the vocabulary at training time, but remains untrained and at its random initialization  since it never occurs in the training data.
we sample a random <unk> vector at test time from the same part of the space as the trained vectors, using a normal distribution with the mean and the variance of M w and the same fixed random seed for all models.