Variable metric or quasi-Newton methods avoid explicit evaluation of the Hessian by building up an approximation of it using successive evaluations of the gradient.
we replace H −1 (θ (k) ) in (5) with a local approximation of the inverse Hessian B (k) : with B (k) a symmatric, positive definite matrix which satisfies the equation: where Variable metric methods also show excellent convergence properties and can be much more efficient than using true Newton updates, but for large scale problems with hundreds of thousands of parameters, even storing the approximate Hessian is prohibitively expensive.