Convex Denoising using Non-Convex Tight Frame Regularization

This letter considers the problem of signal denoising using a sparse tight-frame analysis prior. The l1 norm has been extensively used as a regularizer to promote sparsity; however, it tends to under-estimate non-zero values of the underlying signal. To more accurately estimate non-zero values, we propose the use of a non-convex regularizer, chosen so as to ensure convexity of the objective function. The convexity of the objective function is ensured by constraining the parameter of the non-convex penalty. We use ADMM to obtain a solution and show how to guarantee that ADMM converges to the global optimum of the objective function. We illustrate the proposed method for 1D and 2D signal denoising.

[1]  Patrick L. Combettes,et al.  Proximal Thresholding Algorithm for Minimization over Orthonormal Bases , 2007, SIAM J. Optim..

[2]  Thomas Brox,et al.  An Iterated L1 Algorithm for Non-smooth Non-convex Optimization in Computer Vision , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Jian-Feng Cai,et al.  Split Bregman Methods and Frame Based Image Restoration , 2009, Multiscale Model. Simul..

[4]  Ivan W. Selesnick,et al.  Convex 1-D Total Variation Denoising with Non-convex Regularization , 2015, IEEE Signal Processing Letters.

[5]  Michael Elad,et al.  Analysis versus synthesis in signal priors , 2006, 2006 14th European Signal Processing Conference.

[6]  Donald Geman,et al.  Constrained Restoration and the Recovery of Discontinuities , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  D. Donoho,et al.  Translation-Invariant De-Noising , 1995 .

[8]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[9]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[10]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[11]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[12]  Michael Elad,et al.  On MAP and MMSE estimators for the co-sparse analysis model , 2014, Digit. Signal Process..

[13]  Thomas Brox,et al.  On Iteratively Reweighted Algorithms for Nonsmooth Nonconvex Optimization in Computer Vision , 2015, SIAM J. Imaging Sci..

[14]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[15]  Ivan W. Selesnick,et al.  Artifact-Free Wavelet Denoising: Non-convex Sparse Regularization, Convex Optimization , 2015, IEEE Signal Processing Letters.

[16]  Susanto Rahardja,et al.  Alternating Direction Method for Balanced Image Restoration , 2012, IEEE Transactions on Image Processing.

[17]  Carlo Fischione,et al.  On the Convergence of Alternating Direction Lagrangian Methods for Nonconvex Structured Optimization Problems , 2014, IEEE Transactions on Control of Network Systems.

[18]  Mila Nikolova,et al.  Fast Nonconvex Nonsmooth Minimization Methods for Image Restoration and Reconstruction , 2010, IEEE Transactions on Image Processing.

[19]  Ivan W. Selesnick,et al.  Sparse Signal Estimation by Maximally Sparse Convex Optimization , 2013, IEEE Transactions on Signal Processing.

[20]  Zhi-Quan Luo,et al.  Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems , 2014, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[21]  Hédy Attouch,et al.  Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Lojasiewicz Inequality , 2008, Math. Oper. Res..

[22]  Rabab Kreidieh Ward,et al.  Non-convex row-sparse multiple measurement vector analysis prior formulation for EEG signal reconstruction , 2014, Biomed. Signal Process. Control..

[23]  José M. Bioucas-Dias,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[24]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[25]  Serena Morigi,et al.  Convex Image Denoising via Non-Convex Regularization , 2015, SSVM.

[26]  Javier Portilla,et al.  Image restoration through l0 analysis-based sparse optimization in tight frames , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[27]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[28]  Guoyin Li,et al.  Global Convergence of Splitting Methods for Nonconvex Composite Optimization , 2014, SIAM J. Optim..

[29]  M Bioucas-DiasJosé,et al.  Fast image recovery using variable splitting and constrained optimization , 2010 .

[30]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[31]  S. Osher,et al.  Image restoration: Total variation, wavelet frames, and beyond , 2012 .

[32]  Suchi Saria,et al.  Convex envelopes of complexity controlling penalties: the case against premature envelopment , 2011, AISTATS.

[33]  Mário A. T. Figueiredo,et al.  Signal restoration with overcomplete wavelet transforms: comparison of analysis and synthesis priors , 2009, Optical Engineering + Applications.

[34]  Jian-Feng Cai,et al.  Data-driven tight frame construction and image denoising , 2014 .

[35]  Hong-Kun Xu,et al.  Convergence of Bregman alternating direction method with multipliers for nonconvex composite problems , 2014, 1410.8625.

[36]  Mila Nikolova,et al.  Markovian reconstruction using a GNC approach , 1999, IEEE Trans. Image Process..

[37]  Rick Chartrand,et al.  Fast algorithms for nonconvex compressive sensing: MRI reconstruction from very few data , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[38]  Richard G. Baraniuk,et al.  Fast Alternating Direction Optimization Methods , 2014, SIAM J. Imaging Sci..

[39]  M. Nikolova,et al.  Estimation of binary images by minimizing convex criteria , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[40]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[41]  Richard Baraniuk,et al.  The Dual-tree Complex Wavelet Transform , 2007 .