We apply this idea to motivate two language models: a novel class-based language model and regularized minimum discrimination information (MDI) models.
one can improve exponential language models by adding features (or a prior model) that shrink parameter values while maintaining training performance, and from this observation we develop Heuristics 1 and 2.