Analysis of Sparse Quasi-Newton Updates with Positive Definite Matrix Completion

Quasi-Newton methods are powerful techniques for solving unconstrained minimization problems. Variable metric methods, which include the BFGS and DFP methods, generate dense positive definite approximations and, therefore, are not applicable to large-scale problems. To overcome this difficulty, a sparse quasi-Newton update with positive definite matrix completion that exploits the sparsity pattern $$E :=\{(i, j)\;|\; (\nabla^2 f(x))_{ij} \neq 0\,{\rm for\,some}\, x \in R^n\}$$ of the Hessian is proposed. The proposed method first calculates a partial approximate Hessian $$H^{QN}_{ij}, (i, j) \in F$$ , where $$F \supseteq E$$ , using an existing quasi-Newton update formula such as the BFGS or DFP methods. Next, a full matrix Hk+1, which is a maximum-determinant positive definite matrix completion of $$H^{QN}_{ij}, (i, j) \in F$$ , is obtained. If the sparsity pattern E (or its extension F) has a property related to a chordal graph, then the matrix Hk+1 can be expressed as products of some sparse matrices. The time and space requirements of the proposed method are lower than those of the BFGS or the DFP methods. In particular, when the Hessian matrix is tridiagonal, the complexities become O(n). The proposed method is shown to have superlinear convergence under the usual assumptions.