Modeling human language requires the ability to not only generate fluent text but also encode factual knowledge.
traditional language models are only capable of remembering facts seen at training time, and often have difficulty recalling them.