First, information-theoretic MI (Cover and Thomas, 2006) and its variants (Suzuki et al., 2009;Reshef et al., 2011) are the most commonly used dependence measures.
to the best of our knowledge, there is no practical method of computing MIs for large-multi class high-dimensional (having a complex generative model) discrete data, such as sparse linguistic data.