Slonim (2002) claims that the mutual information I(X;X) measures the compactness of the new representation.
the ratedistortion function is a trade-off between the compactness of new representation and the expected distortion.