A standard statistical language model computes the probability of a word sequence W = w 1 , w 2 , ..., w T as a product of conditional probabilities of each word w i given its history, which is typically approximated by just one or two preceding words (leading to bigrams, and trigrams, respectively).
a trigram language model is described by Even with this limitation, the estimation of the required probabilities is challenging: many word contexts may be observed infrequently or not at all, leading to unreliable probability estimates under maximum likelihood estimation.