The mutual information (MI) 3 between two random variables X and Y is defined by (Cover and Thomas, 2006), where KL[• •] denotes the Kullback-Leibler (KL) divergence.
mI(X, Y ) is the degree of dependence between X and Y measured by the KL divergence between P XY and P X P Y .