We are interested in estimating the posterior p(θ, z | x).
variational inference transforms this problem into an optimization problem where we try to find a distribution q(θ, z) from a restricted set Q that minimizes the KL-divergence between q(θ, z) and p(θ, z | x): Rearranging the above yields: where F is defined as F is a lower bound on likelihood.