As always, there is a trade-off between accuracy, space, and time, with recent papers considering small but approximate lossy LMs (Chazelle et al., 2004;Talbot and Osborne, 2007;Guthrie and Hepple, 2010), or loss-less LMs backed by tries (Stolcke et al., 2011), or related compressed structures (Germann et al., 2009;Heafield, 2011;Pauls and Klein, 2011;Sorensen and Allauzen, 2011;Watanabe et al., 2009).
none of these approaches scale well to very high-order m or very large corpora, due to their high memory and time requirements.