A superlinearly convergent exact penalty method for constrained nonlinear least squares: global analysis

We have recently proposed a structured algorithm for solving constrained nonlinear least-squares problems and established its local two-step Q-superlinear convergence rate. The approach is based on an earlier adaptive structured scheme due to Mahdavi-Amiri and Bartels of the exact penalty method. The structured adaptation makes use of the ideas of Nocedal and Overton for handling quasi-Newton updates of projected Hessians and adapts a structuring scheme due to Engels and Martinez. For robustness, we have employed a specific nonsmooth line search strategy, taking account of the least-squares objective. Numerical results also confirm the practical relevance of our special considerations for the inherent structure of the least squares. Here, we establish global convergence of the proposed algorithm using a weaker condition than the one used by the exact penalty method of Coleman and Conn for general nonlinear programs.

[1]  T. Pietrzykowski.  An Exact Potential Method for Constrained Maxima , 1969 .

[2]  T. Pietrzykowski,et al.  A Penalty Function Method Converging Directly to a Constrained Optimum , 1977 .

[3]  P. Gill,et al.  Algorithms for the Solution of the Nonlinear Least-Squares Problem , 1978 .

[4]  Klaus Schittkowski,et al.  Test examples for nonlinear programming codes , 1980 .

[5]  Andrew R. Conn,et al.  Second-order conditions for an exact penalty function , 1980, Math. Program..

[6]  John E. Dennis,et al.  An Adaptive Nonlinear Least-Squares Algorithm , 1977, TOMS.

[7]  Andrew R. Conn,et al.  Nonlinear programming via an exact penalty function: Global analysis , 1982, Math. Program..

[8]  Andrew R. Conn,et al.  Nonlinear programming via an exact penalty function: Asymptotic analysis , 1982, Math. Program..

[9]  T. Coleman,et al.  On the Local Convergence of a Quasi-Newton Method for the Nonlinear Programming Problem , 1984 .

[10]  Senad Busovaca.  Handling degeneracy in a nonlinear l(,1) algorithm , 1985 .

[11]  R. Bartels,et al.  On generating test problems for nonlinear programming algorithms , 1986 .

[12]  R. Fletcher,et al.  Hybrid Methods for Nonlinear Least Squares , 1987 .

[13]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[14]  R. Tapia.  On secant updates for use in general constrained optimization , 1988 .

[15]  Richard H. Bartels,et al.  Constrained nonlinear least squares: an exact penalty approach with projected structured quasi-Newton updates , 1989, TOMS.

[16]  J. Dennis,et al.  Convergence theory for the structured BFGS secant method with an application to nonlinear least squares , 1989 .

[17]  Jorge Nocedal,et al.  An analysis of reduced Hessian methods for constrained optimization , 1991, Math. Program..

[18]  L. Biegler,et al.  Simultaneous solution and optimization strategies for parameter estimation of differential-algebraic equation systems , 1991 .

[19]  Héctor J. Martínez,et al.  Local and Superlinear Convergence for Partially Known Quasi-Newton Methods , 1991, SIAM J. Optim..

[20]  J. Huschens.  Exploiting additional structure in equality constrained optimization by structured SQP secant algorithms , 1993 .

[21]  I. Duff,et al.  The state of the art in numerical analysis , 1997 .

[22]  Michael A. Saunders,et al.  SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization , 2002, SIAM J. Optim..

[23]  Nicholas I. M. Gould,et al.  An algorithm for nonlinear optimization using linear programming and equality constrained subproblems , 2004, Math. Program..

[24]  Mihai Anitescu,et al.  Global Convergence of an Elastic Mode Approach for a Class of Mathematical Programs with Complementarity Constraints , 2005, SIAM J. Optim..

[25]  Nicholas I. M. Gould,et al.  On the Convergence of Successive Linear-Quadratic Programming Algorithms , 2005, SIAM J. Optim..

[26]  Jorge Nocedal,et al.  Interior Methods for Mathematical Programs with Complementarity Constraints , 2006, SIAM J. Optim..

[27]  Robert J. Vanderbei,et al.  Interior-Point Algorithms, Penalty Methods and Equilibrium Problems , 2006, Comput. Optim. Appl..

[28]  Jorge Nocedal,et al.  Knitro: An Integrated Package for Nonlinear Optimization , 2006 .

[29]  N. Mahdavi-Amiri,et al.  A Superlinearly Convergent Penalty Method with Nonsmooth Line Search for Constrained Nonlinear Least Squares , 2012 .

[30]  N. Mahdavi-Amiri,et al.  SUPERLINEARLY CONVERGENT EXACT PENALTY PROJECTED STRUCTURED HESSIAN UPDATING SCHEMES FOR CONSTRAINED NONLINEAR LEAST SQUARES: ASYMPTOTIC ANALYSIS , 2012 .