Note that datasets with different vocabulary size can lead to different entropy: the entropy of picking a word from the vocabulary uniformly at random would have been different.
consider a stimulus-response pair, where the stimulus is a sequence of tokens s = (s 1 , s 2 , ..., s m ), and the response is a sequence of tokens r = (r 1 , r 2 , ..., r n ).