Thus, for α < 1, H α (q) is an upper bound on the Shannon entropy.
since l α regularization minimizes the Renyi α-entropy, which for 0 < α < 1 is an upper bound on the shannon entropy, it effectively maximizes the mutual information between c i and c i−1 , given that we have shown that at least in the context of clustering, sparsifying both the observation probabilities and the state transition probabilities of an HMM using the l α prior directly minimizes generalization error.