It has been proposed in (Hoffman and Johnson, 2016) that decomposing the ELBO can produce an equivalent way to define the ELBO: where q φ (x, z) denotes the variational joint distribution induced by the posterior denotes the mutual information between x and z under q φ (x, z), and denotes the aggregated posterior (Makhzani et al., 2016).
the zero-forcing effect of the KL indicates if the latent variable z is a good representation of x, the mutual information between x and z should take a large value and the KL term in Equation (3) would be non-zero.