A Two-Stage Active-Set Algorithm for Bound-Constrained Optimization

In this paper, we describe a two-stage method for solving optimization problems with bound constraints. It combines the active-set estimate described in Facchinei and Lucidi (J Optim Theory Appl 85(2):265–289, 1995) with a modification of the non-monotone line search framework recently proposed in De Santis et al. (Comput Optim Appl 53(2):395–423, 2012). In the first stage, the algorithm exploits a property of the active-set estimate that ensures a significant reduction in the objective function when setting to the bounds all those variables estimated active. In the second stage, a truncated-Newton strategy is used in the subspace of the variables estimated non-active. In order to properly combine the two phases, a proximity check is included in the scheme. This new tool, together with the other theoretical features of the two stages, enables us to prove global convergence. Furthermore, under additional standard assumptions, we can show that the algorithm converges at a superlinear rate. Promising experimental results demonstrate the effectiveness of the proposed method.

[1]  Nicholas I. M. Gould,et al.  GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization , 2003, TOMS.

[2]  Chih-Jen Lin,et al.  Newton's Method for Large Bound-Constrained Optimization Problems , 1999, SIAM J. Optim..

[3]  Nicholas I. M. Gould,et al.  CUTEst: a Constrained and Unconstrained Testing Environment with safe threads for mathematical optimization , 2013, Computational Optimization and Applications.

[4]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[5]  Christian Kanzow,et al.  On Affine-Scaling Interior-Point Newton Methods for Nonlinear Minimization with Bound Constraints , 2006, Comput. Optim. Appl..

[6]  Ernesto G. Birgin,et al.  2 . 2 Meaning of “ to solve a problem ” , 2011 .

[7]  P. Toint,et al.  Global convergence of a class of trust region algorithms for optimization with simple bounds , 1988 .

[8]  S. Lucidi,et al.  Quadratically and superlinearly convergent algorithms for the solution of inequality constrained minimization problems , 1995 .

[9]  José Mario Martínez,et al.  Second-order negative-curvature methods for box-constrained and general constrained optimization , 2010, Comput. Optim. Appl..

[10]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[11]  L. Grippo,et al.  A class of continuously differentiable exact penalty function algorithms for nonlinear programming problems , 1984 .

[12]  Francisco Facchinei,et al.  An Active Set Newton Algorithm for Large-Scale Nonlinear Programs with Box Constraints , 1998, SIAM J. Optim..

[13]  L. Grippo,et al.  A differentiable exact penalty function for bound constrained quadratic programming problems , 1991 .

[14]  Donghui Li,et al.  An Active Set Modified Polak–Ribiére–Polyak Method for Large-Scale Nonlinear Bound Constrained Optimization , 2012, Journal of Optimization Theory and Applications.

[15]  José Mario Martínez,et al.  Large-Scale Active-Set Box-Constrained Optimization Method with Spectral Projected Gradients , 2002, Comput. Optim. Appl..

[16]  L. N. Vicente,et al.  Trust-Region Interior-Point SQP Algorithms for a Class of Nonlinear Programming Problems , 1998 .

[17]  William W. Hager,et al.  A New Active Set Algorithm for Box Constrained Optimization , 2006, SIAM J. Optim..

[18]  William W. Hager,et al.  A Nonmonotone Line Search Technique and Its Application to Unconstrained Optimization , 2004, SIAM J. Optim..

[19]  E. Polak,et al.  Family of Projected Descent Methods for Optimization Problems with Simple Bounds , 1997 .

[20]  Gianni Di Pillo,et al.  An active set feasible method for large-scale minimization problems with bound constraints , 2012, Computational Optimization and Applications.

[21]  Stefan Ulbrich,et al.  Superlinear and quadratic convergence of affine-scaling interior-point Newton methods for problems with simple bounds without strict complementarity assumption , 1999, Math. Program..

[22]  D. Bertsekas.  Projected Newton methods for optimization problems with simple constraints , 1981, CDC 1981.

[23]  Stefano Lucidi,et al.  A Fast Active Set Block Coordinate Descent Algorithm for ℓ1-Regularized Least Squares , 2014, SIAM J. Optim..

[24]  L. Grippo,et al.  A class of nonmonotone stabilization methods in unconstrained optimization , 1991 .

[25]  Christoph Buchheim,et al.  A Feasible Active Set Method with Reoptimization for Convex Quadratic Mixed-Integer Programming , 2015, SIAM J. Optim..

[26]  Francisco Facchinei,et al.  A Truncated Newton Algorithm for Large Scale Box Constrained Optimization , 2002, SIAM J. Optim..

[27]  Trond Steihaug,et al.  Truncated-newtono algorithms for large-scale unconstrained optimization , 1983, Math. Program..