To make matters worse, language is Zipf-distributed, so that a large fraction of any training data set will be hapax legomena, very many word types will appear only a few times, and many word types will be left out of the training set altogether.
for many word types supervised NLP systems have very few, or even zero, labeled examples from which to estimate parameters.