Since the training of language models needs nothing but the raw text, it has almost unlimited corpora.
conducting training on extensive corpora results in a huge dictionary, and makes calculating the vanilla softmax intractable.