An active-set algorithmic framework for non-convex optimization problems over the simplex

In this paper, we describe a new active-set algorithmic framework for minimizing a non-convex function over the unit simplex. At each iteration, the method makes use of a rule for identifying active variables (i.e., variables that are zero at a stationary point) and specific directions (that we name active-set gradient related directions) satisfying a new "nonorthogonality" type of condition. We prove global convergence to stationary points when using an Armijo line search in the given framework. We further describe three different examples of active-set gradient related directions that guarantee linear convergence rate (under suitable assumptions). Finally, we report numerical experiments showing the effectiveness of the approach.

[1]  W. Hager,et al.  An active set algorithm for nonlinear optimization with polyhedral constraints , 2016, 1606.01992.

[2]  Christoph Buchheim,et al.  A Feasible Active Set Method with Reoptimization for Convex Quadratic Mixed-Integer Programming , 2015, SIAM J. Optim..

[3]  Gianni Di Pillo,et al.  An active set feasible method for large-scale minimization problems with bound constraints , 2012, Computational Optimization and Applications.

[4]  William W. Hager,et al.  Projection onto a Polyhedron that Exploits Sparsity , 2016, SIAM J. Optim..

[5]  Patrice Marcotte,et al.  Some comments on Wolfe's ‘away step’ , 1986, Math. Program..

[6]  Andreas Fischer,et al.  A block active set algorithm with spectral choice line search for the symmetric eigenvalue complementarity problem , 2017, Appl. Math. Comput..

[7]  Kenneth L. Clarkson,et al.  Coresets, sparse greedy approximation, and the Frank-Wolfe algorithm , 2008, SODA '08.

[8]  Philip Wolfe,et al.  An algorithm for quadratic programming , 1956 .

[9]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[10]  J. Jeffry Howbert,et al.  The Maximum Clique Problem , 2007 .

[11]  Etienne de Klerk,et al.  The complexity of optimizing over a simplex, hypercube or sphere: a short survey , 2008, Central Eur. J. Oper. Res..

[12]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[13]  S. Lucidi,et al.  Quadratically and superlinearly convergent algorithms for the solution of inequality constrained minimization problems , 1995 .

[14]  Daniela di Serafino,et al.  A Two-Phase Gradient Method for Quadratic Programming Problems with a Single Linear Constraint and Bounds on the Variables , 2017, SIAM J. Optim..

[15]  William W. Hager,et al.  A New Active Set Algorithm for Box Constrained Optimization , 2006, SIAM J. Optim..

[16]  Jie Sun,et al.  Solution Methodologies for the Smallest Enclosing Circle Problem , 2003, Comput. Optim. Appl..

[17]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[18]  Stefano Lucidi,et al.  A Fast Active Set Block Coordinate Descent Algorithm for ℓ1-Regularized Least Squares , 2014, SIAM J. Optim..

[19]  Francisco Facchinei,et al.  On the Accurate Identification of Active Constraints , 1998, SIAM J. Optim..

[20]  J. J. Moré,et al.  Algorithms for bound constrained quadratic programming problems , 1989 .

[21]  Martin Jaggi,et al.  On the Global Linear Convergence of Frank-Wolfe Optimization Variants , 2015, NIPS.

[22]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[23]  Stefano Lucidi,et al.  A Two-Stage Active-Set Algorithm for Bound-Constrained Optimization , 2016, Journal of Optimization Theory and Applications.

[24]  Francisco Facchinei,et al.  An Active Set Newton Algorithm for Large-Scale Nonlinear Programs with Box Constraints , 1998, SIAM J. Optim..

[25]  José Mario Martínez,et al.  Large-Scale Active-Set Box-Constrained Optimization Method with Spectral Projected Gradients , 2002, Comput. Optim. Appl..

[26]  P. Tseng,et al.  On the linear convergence of descent methods for convex essentially smooth minimization , 1992 .

[27]  Martin Jaggi,et al.  An Affine Invariant Linear Convergence Analysis for Frank-Wolfe Algorithms , 2013, 1312.7864.

[28]  Alfredo N. Iusem,et al.  Splitting methods for the Eigenvalue Complementarity Problem , 2018, Optim. Methods Softw..

[29]  D. Bertsekas.  Projected Newton methods for optimization problems with simple constraints , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[30]  L. Grippo,et al.  A class of continuously differentiable exact penalty function algorithms for nonlinear programming problems , 1984 .

[31]  Immanuel M. Bomze,et al.  Evolution towards the Maximum Clique , 1997, J. Glob. Optim..