Complementarity Formulations of ' 0 -norm Optimization Problems

In a number of application areas, it is desirable to obtain sparse solutions. Minimizing the number of nonzeroes of the solution (its ‘0-norm) is a dicult nonconvex optimization problem, and is often approximated by the convex problem of minimizing the ‘1-norm. In contrast, we consider exact formulations as mathematical programs with complementarity constraints and their reformulations as smooth nonlinear programs. We discuss properties of the various formulations and their connections to the original ‘0-minimization problem in terms of stationarity conditions, as well as local and global optimality. Numerical experiments using randomly generated problems show that standard nonlinear programming solvers, applied to the smooth but nonconvex equivalent reformulations, are often able to nd sparser solutions than those obtained by the convex ‘1-approximation.

[1]  Jorge Nocedal,et al.  Knitro: An Integrated Package for Nonlinear Optimization , 2006 .

[2]  Claudio Gentile,et al.  Projected Perspective Reformulations with Applications in Design Problems , 2011, Oper. Res..

[3]  Bethany L. Nicholson,et al.  Mathematical Programs with Equilibrium Constraints , 2021, Pyomo — Optimization Modeling in Python.

[4]  Jing Hu,et al.  On the Global Solution of Linear Programs with Linear Complementarity Constraints , 2008, SIAM J. Optim..

[5]  Oktay Günlük,et al.  Perspective reformulations of mixed integer nonlinear programs with indicator variables , 2010, Math. Program..

[6]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[7]  R. Andreani,et al.  Constant-Rank Condition and Second-Order Constraint Qualification , 2010 .

[8]  Christian Kanzow,et al.  A New Regularization Method for Mathematical Programs with Complementarity Constraints with Strong Convergence Properties , 2013, SIAM J. Optim..

[9]  Stefan Scholtes,et al.  Convergence Properties of a Regularization Scheme for Mathematical Programs with Complementarity Constraints , 2000, SIAM J. Optim..

[10]  Duan Li,et al.  Successive convex approximations to cardinality-constrained convex programs: a piecewise-linear DC approach , 2014, Comput. Optim. Appl..

[11]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[12]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[13]  Sven Leyffer,et al.  Solving mathematical programs with complementarity constraints as nonlinear programs , 2004, Optim. Methods Softw..

[14]  Michael A. Saunders,et al.  SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization , 2002, SIAM J. Optim..

[15]  Jorge Nocedal,et al.  Interior Methods for Mathematical Programs with Complementarity Constraints , 2006, SIAM J. Optim..

[16]  Christian Kanzow,et al.  The Price of Inexactness: Convergence Properties of Relaxation Methods for Mathematical Programs with Complementarity Constraints Revisited , 2015, Math. Oper. Res..

[17]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[18]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[19]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[20]  Yin Zhang,et al.  Theory of Compressive Sensing via ℓ1-Minimization: a Non-RIP Analysis and Extensions , 2013 .

[22]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[23]  Jing Hu,et al.  On linear programs with linear complementarity constraints , 2011, Journal of Global Optimization.

[24]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[25]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[26]  Michael P. Friedlander,et al.  A two-sided relaxation scheme for Mathematical Programs with Equilibrium Constraints , 2005, SIAM J. Optim..

[27]  R. Janin.  Directional derivative of the marginal function in nonlinear programming , 1984 .

[28]  Arne Drud,et al.  CONOPT: A GRG code for large sparse dynamic nonlinear optimization problems , 1985, Math. Program..

[29]  Christian Kanzow,et al.  On a Reformulation of Mathematical Programs with Cardinality Constraints , 2015 .

[30]  Olvi L. Mangasarian,et al.  Hybrid misclassification minimization , 1996, Adv. Comput. Math..

[31]  Brian W. Kernighan,et al.  AMPL: A Modeling Language for Mathematical Programming , 1993 .

[32]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[33]  Trevor J. Hastie,et al.  Genome-wide association analysis by lasso penalized logistic regression , 2009, Bioinform..

[34]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[35]  Olvi L. Mangasarian,et al.  Misclassification minimization , 1994, J. Glob. Optim..

[36]  E. Candès,et al.  Error correction via linear programming , 2005, FOCS 2005.

[37]  Stefan Scholtes,et al.  Nonconvex Structures in Nonlinear Programming , 2004, Oper. Res..

[38]  Yonina C. Eldar,et al.  Introduction to Compressed Sensing , 2022 .