In this paper we focus on a class of LMs, which we will call mixture of distributions LMs (MODLMs; §2).
we define MODLMs as all LMs that take the following form, calculating the probabilities of the next word in a sentence w i given preceding context c according to a mixture of several component probability distributions P k (w i |c): Here, λ k (c) is a function that defines the mixture weights, with the constraint that K k=1 λ k (c) = 1 for all c. This form is not new in itself, and widely used both in the calculation of smoothing coefficients for n-gram LMs (Chen and Goodman, 1996), and interpolation of LMs of various varieties (Jelinek and Mercer, 1980).