Some properties of the Hessian of the logarithmic barrier function

More than twenty years ago, Murray and Lootsma showed that Hessian matrices of the logarithmic barrier function become increasingly ill-conditioned at points on the barrier trajectory as the solution is approached. This paper explores some further characteristics of the barrier Hessian. We first show that, except in two special cases, the barrier Hessian is ill-conditioned in an entire region near the solution. At points in a more restricted region (including the barrier trajectory itself), this ill-conditioning displays a special structure connected with subspaces defined by the Jacobian of the active constraints. We then indicate how a Cholesky factorization with diagonal pivoting can be used to detect numerical rank-deficiency in the barrier Hessian, and to provide information about the underlying subspaces without making an explicit prediction of the active constraints. Using this subspace information, a close approximation to the Newton direction can be calculated by solving linear systems whose condition reflects that of the original problem.

[1]  J. H. Wilkinson The algebraic eigenvalue problem , 1966 .

[2]  Anthony V. Fiacco,et al.  Nonlinear programming;: Sequential unconstrained minimization techniques , 1968 .

[3]  M. J. D. Powell,et al.  Nonlinear Programming—Sequential Unconstrained Minimization Techniques , 1969 .

[4]  W. Murray,et al.  Analytical expressions for the eigenvalues and eigenvectors of the Hessian matrices of barrier and penalty functions , 1971 .

[5]  M. H. Wright Numerical methods for nonlinearly constrained optimization , 1976 .

[6]  W. Murray,et al.  Projected Lagrangian Methods Based on the Trajectories of Penalty and Barrier Functions. , 1978 .

[7]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[8]  Philip E. Gill,et al.  Practical optimization , 1981 .

[9]  Gene H. Golub,et al.  Matrix computations , 1983 .

[10]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[11]  Danny C. Sorensen,et al.  A note on the computation of an orthonormal basis for the null space of a matrix , 1982, Math. Program..

[12]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, STOC '84.

[13]  R. Fletcher Practical Methods of Optimization , 1988 .

[14]  N. Higham Analysis of the Cholesky Decomposition of a Semi-definite Matrix , 1990 .

[15]  G. Stewart,et al.  Matrix Perturbation Theory , 1990 .

[16]  Elizabeth Eskow,et al.  A New Modified Cholesky Factorization , 1990, SIAM J. Sci. Comput..

[17]  Margaret H. Wright,et al.  Interior methods for constrained optimization , 1992, Acta Numerica.

[18]  Clóvis C. Gonzaga,et al.  Path-Following Methods for Linear Programming , 1992, SIAM Rev..

[19]  Gautam Appa,et al.  Numerical Linear Algebra and Optimization: Volume 1 , 1992 .

[20]  Stephen G. Nash,et al.  A Barrier Method for Large-Scale Constrained Optimization , 1993, INFORMS J. Comput..