For this reason, a context larger than that provided by a trigram is typically required, which quickly leads to data-sparsity.
to these approaches, factored language models encode morphological knowledge not by altering the linear segmentation of words but by encoding words as parallel bundles of features.