In order to better understand how different modifications of the soft label loss affected model performance on transfer tasks, we conducted some additional experiments.
we investigated the effects of (a) varying the number N of word neighbors used to compute the loss function, and (b) removing the normalization factor by getting rid of the denominator in Eq.