A Unified Framework for Solving a General Class of Nonconvexly Regularized Convex Models

Recently, several nonconvex sparse regularizers which can preserve the convexity of the cost function have received increasing attention. This paper proposes a general class of such convexity-preserving (CP) regularizers, termed partially smoothed difference-of-convex (pSDC) regularizer. The pSDC regularizer is formulated as a structured difference-of-convex (DC) function, where the landscape of the subtrahend function can be adjusted by a parameterized smoothing function so as to attain overall-convexity. Assigned with proper building blocks, the pSDC regularizer reproduces existing CP regularizers and opens the way to a large number of promising new ones. With respect to the resultant nonconvexly regularized convex (NRC) model, we derive a series of overall-convexity conditions which naturally embrace the conditions in previous works. Moreover, we develop a unified framework based on DC programming for solving the NRC model. Compared to previously reported proximal splitting type approaches, the proposed framework makes less stringent assumptions. We establish the convergence of the proposed framework to a global minimizer. Numerical experiments demonstrate the power of the pSDC regularizers and the efficiency of the proposed DC algorithm.

[1]  I. Yamada,et al.  An inexact proximal linearized DC algorithm with provably terminating inner loop , 2023, Optimization.

[2]  I. Yamada,et al.  A Unified Class of DC-type Convexity-Preserving Regularizers for Improved Sparse Regularization , 2022, 2022 30th European Signal Processing Conference (EUSIPCO).

[3]  I. Yamada,et al.  DC-LiGME: An Efficient Algorithm for Improved Convex Sparse Regularization , 2021, Asilomar Conference on Signals, Systems and Computers.

[4]  I. Selesnick,et al.  Non-convex Total Variation Regularization for Convex Denoising of Signals , 2020, Journal of Mathematical Imaging and Vision.

[5]  I. Yamada,et al.  Linearly involved generalized Moreau enhanced models and their proximal splitting algorithm under overall convexity condition , 2019, Inverse Problems.

[6]  Isao Yamada,et al.  Convexity-edge-preserving Signal Recovery with Linearly Involved Generalized Minimax Concave Penalty Function , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Michel P. Tcheou,et al.  An Inertial Algorithm for DC Programming , 2018, Set-Valued and Variational Analysis.

[8]  Zhe Sun,et al.  Enhanced proximal DC algorithms with extrapolation for a class of structured nonsmooth DC minimization , 2018, Mathematical Programming.

[9]  Le Thi Hoai An,et al.  Convergence Analysis of Difference-of-Convex Algorithm with Subanalytic Data , 2018, Journal of Optimization Theory and Applications.

[10]  Le Thi Hoai An,et al.  DC programming and DCA: thirty years of developments , 2018, Math. Program..

[11]  Amir Beck,et al.  First-Order Methods in Optimization , 2017 .

[12]  Ivan W. Selesnick,et al.  Sparse Regularization via Convex Analysis , 2017, IEEE Transactions on Signal Processing.

[13]  Prabhu Babu,et al.  Majorization-Minimization Algorithms in Signal Processing, Communications, and Machine Learning , 2017, IEEE Transactions on Signal Processing.

[14]  Antoine Soubeyran,et al.  Global convergence of a proximal linearized algorithm for difference of convex functions , 2015, Optimization Letters.

[15]  Ivan W. Selesnick,et al.  Enhanced Low-Rank Matrix Approximation , 2015, IEEE Signal Processing Letters.

[16]  Damek Davis,et al.  A Three-Operator Splitting Scheme and its Optimization Applications , 2015, 1504.01032.

[17]  Le Thi Hoai An,et al.  DC approximation approaches for sparse optimization , 2014, Eur. J. Oper. Res..

[18]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[19]  Ivan W. Selesnick,et al.  Total variation denoising with overlapping group sparsity , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[20]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[21]  Marc Teboulle,et al.  Smoothing and First Order Methods: A Unified Framework , 2012, SIAM J. Optim..

[22]  F. Bach,et al.  Optimization with Sparsity-Inducing Penalties (Foundations and Trends(R) in Machine Learning) , 2011 .

[23]  Julien Mairal,et al.  Optimization with Sparsity-Inducing Penalties , 2011, Found. Trends Mach. Learn..

[24]  Homer F. Walker,et al.  Anderson Acceleration for Fixed-Point Iterations , 2011, SIAM J. Numer. Anal..

[25]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[26]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[27]  Xuecheng Tai,et al.  AUGMENTED LAGRANGIAN METHOD FOR TOTAL VARIATION RESTORATION WITH NON-QUADRATIC FIDELITY , 2011 .

[28]  Jean-Philippe Vert,et al.  Fast detection of multiple change-points shared by many signals using group LARS , 2010, NIPS.

[29]  Cun-Hui Zhang Nearly unbiased variable selection under minimax concave penalty , 2010, 1002.4734.

[30]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[31]  Yonina C. Eldar,et al.  Block-Sparse Signals: Uncertainty Relations and Efficient Recovery , 2009, IEEE Transactions on Signal Processing.

[32]  Ron Meir,et al.  A bilinear formulation for vector sparsity optimization , 2008, Signal Process..

[33]  Y. Censor,et al.  The multiple-sets split feasibility problem and its applications for inverse problems , 2005 .

[34]  E. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[35]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[36]  R. Horst,et al.  DC Programming: Overview , 1999 .

[37]  Mila Nikolova,et al.  Markovian reconstruction using a GNC approach , 1999, IEEE Trans. Image Process..

[38]  M. Nikolova,et al.  Estimation of binary images by minimizing convex criteria , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[39]  Jeffrey A. Fessler,et al.  Grouped coordinate descent algorithms for robust edge-preserving image restoration , 1997, Optics & Photonics.

[40]  Ken D. Sauer,et al.  A unified approach to statistical tomography using coordinate descent optimization , 1996, IEEE Trans. Image Process..

[41]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[42]  A constrained LiGME model and its proximal splitting algorithm under overall convexity condition , 2022, Journal of Applied and Numerical Optimization.

[43]  I. Selesnick,et al.  Sharpening Sparse Regularizers via Smoothing , 2021, IEEE Open Journal of Signal Processing.

[44]  Masahiro Yukawa,et al.  Robust Recovery of Jointly-Sparse Signals Using Minimax Concave Loss Function , 2021, IEEE Transactions on Signal Processing.

[45]  Gitta Kutyniok Compressed Sensing , 2012 .

[46]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[47]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[48]  T. P. Dinh,et al.  Convex analysis approach to d.c. programming: Theory, Algorithm and Applications , 1997 .

[49]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[50]  K. Lange Convergence of EM image reconstruction algorithms with Gibbs smoothing. , 1990, IEEE transactions on medical imaging.

[51]  Y. Nesterov A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .