Neural language models are typically trained by predicting the next word given a past context (Bengio et al., 2003).
natural sentences are not constructed as simple linear word sequences, as they usually contain complex syntactic information.