But it will not predict very well certain locally regular words like of, the etc whose main role is to support the syntactic structure in a sentence.
n-gram language models are able to model them well because of maximum likelihood estimation from training corpus and various smoothing techniques.