Interestingly, we observe F 2 > G 2 > F 3 (this trend holds for larger values of N , omitted here for clarity).
knowing both the preceding N − 1 tokens and tokens in the stimulus results lowers the uncertainty over the next token in response (bigram+stimulus entropy < bigram entropy); on the other hand, this is not as effective as knowing one more token in the preceding block (trigram entropy < bigram+stimulus entropy).