Distributional models (DS: Turney and Pantel (2010); Clark (2012); Erk (2012)), and in particular neural network approaches (Bengio et al., 2003;Collobert et al., 2011;Huang et al., 2012;Mikolov et al., 2013), do not fare well in the absence of large corpora.
for a DS model to learn a word vector, it must have seen that word a sufficient number of times.