Optimization problems involving group sparsity terms

This paper studies a general form problem in which a lower bounded continuously differentiable function is minimized over a block separable set incorporating a group sparsity expression as a constraint or a penalty (or both) in the group sparsity setting. This class of problems is generally hard to solve, yet highly applicable in numerous practical settings. Particularly, we study the proximal mapping that includes group-sparsity terms, and derive an efficient method to compute it. Necessary optimality conditions for the problem are devised, and a hierarchy between stationary-based and coordinate-wised based conditions is established. Methods that obtain points satisfying the optimality conditions are presented, analyzed and tested in applications from the fields of investment and graph theory.

[1]  Volkan Cevher,et al.  Model-based compressive sensing for signal ensembles , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[2]  Panos M. Pardalos,et al.  The maximum clique problem , 1994, J. Glob. Optim..

[3]  Amir Beck,et al.  The Sparse Principal Component Analysis Problem: Optimality Conditions and Algorithms , 2015, Journal of Optimization Theory and Applications.

[4]  A. Rinaldo,et al.  On the asymptotic properties of the group lasso estimator for linear models , 2008 .

[5]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[6]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[7]  Yonina C. Eldar,et al.  Structured Compressed Sensing: From Theory to Applications , 2011, IEEE Transactions on Signal Processing.

[8]  Babak Hassibi,et al.  On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements , 2008, IEEE Transactions on Signal Processing.

[9]  J. Hammersley.  SIMULATION AND THE MONTE CARLO METHOD , 1982 .

[10]  Francis R. Bach,et al.  Structured Variable Selection with Sparsity-Inducing Norms , 2009, J. Mach. Learn. Res..

[11]  Nadav Hallak,et al.  Proximal Mapping for Symmetric Penalty and Sparsity , 2018, SIAM J. Optim..

[12]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[13]  Dirk P. Kroese,et al.  Simulation and the Monte Carlo Method (Wiley Series in Probability and Statistics) , 1981 .

[14]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[15]  Jean-Yves Audibert.  Optimization for Machine Learning , 1995 .

[16]  Inderjit S. Dhillon,et al.  Structured Sparse Regression via Greedy Hard Thresholding , 2016, NIPS.

[17]  Jean-Luc Prigent,et al.  Portfolio Optimization and Performance Analysis , 2007 .

[18]  Yonina C. Eldar,et al.  Sparsity Constrained Nonlinear Optimization: Optimality Conditions and Algorithms , 2012, SIAM J. Optim..

[19]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[20]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[21]  Nadav Hallak,et al.  On the Minimization Over Sparse Symmetric Sets: Projections, Optimality Conditions, and Algorithms , 2016, Math. Oper. Res..

[22]  Fred W. Glover,et al.  The unconstrained binary quadratic programming problem: a survey , 2014, Journal of Combinatorial Optimization.

[23]  Volkan Cevher,et al.  Group-Sparse Model Selection: Hardness and Relaxations , 2013, IEEE Transactions on Information Theory.

[24]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[25]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[26]  Amir Beck,et al.  Introduction to Nonlinear Optimization - Theory, Algorithms, and Applications with MATLAB , 2014, MOS-SIAM Series on Optimization.

[27]  Dirk P. Kroese,et al.  Simulation and the Monte Carlo method , 1981, Wiley series in probability and mathematical statistics.

[28]  Yonina C. Eldar,et al.  Introduction to Compressed Sensing , 2022 .

[29]  Stephen J. Wright,et al.  Computational Methods for Sparse Solution of Linear Inverse Problems , 2010, Proceedings of the IEEE.

[30]  P. Bühlmann,et al.  The group lasso for logistic regression , 2008 .

[31]  Yonina C. Eldar,et al.  Block-Sparse Signals: Uncertainty Relations and Efficient Recovery , 2009, IEEE Transactions on Signal Processing.

[32]  Mike E. Davies,et al.  Sampling Theorems for Signals From the Union of Finite-Dimensional Linear Subspaces , 2009, IEEE Transactions on Information Theory.