The decrease in H(X|X n ) globally indicates that given an n-length sequence x n and another (n + 1)-length sequence y n+1 , the following inequality holds on average: One reason why inequality (2) holds for language data is that there is context in language, and y n+1 carries a longer context as compared with x n .
if we suppose that x n is the prefix of x n+1 , then it is very likely that holds, because the longer the preceding n-gram, the longer the same context.