Note that f (x, y) is the bigram frequency, |W i−1 | the sequence length of W i−1 , and is the difference between the empirical Shannon entropy measured on W i and W i−1 , using maximum likelihood estimates.
this empirical esti-mateH(W ) for a sequence W corresponds to: For this equation to work, one needs to estimate other model parameters.