Assuming that our model has K topics, and once again using z i as the topic assignment for token w i , we consider two partitions S = {S 1 , ..., S K } and T = {T 1 , ..., T K } of the set of tokens w, such that S i = {w j | z j = i} and Variation of information is where H z [•] is entropy with respect to topic distribution and MI (S, T ) is the mutual information between S and T .
we measure how much information we lose in our topic assignments if we reassign every token to the topic of the token that follows.