If we assume a priori that the length of a block is unbounded, then this state space is N K where 0 ∈ N. The probability of transitioning from a state b to another stateb ∈ N K is: where N = kb k , ζ N is the probability that a block has N tokens, 2 and π(b) is the transition distribution given a vector b.
we extend this approach to allow each block of text to be a mixture of multiple classes.