A Majorize-Minimize Subspace Approach for ℓ2-ℓ0 Image Regularization

In this work, we consider a class of differentiable criteria for sparse image computing problems, where a nonconvex regularization is applied to an arbitrary linear transform of the target image. As special cases, it includes edge-preserving measures or frame-analysis potentials commonly used in image processing. As shown by our asymptotic results, the l2-l0 penalties we consider may be employed to provide approximate solutions to l0-penalized optimization problems. One of the advantages of the proposed approach is that it allows us to derive an efficient Majorize-Minimize subspace algorithm. The convergence of the algorithm is investigated by using recent results in nonconvex optimization. The fast convergence properties of the proposed optimization method are illustrated through image processing examples. In particular, its effectiveness is demonstrated on several data recovery problems.

[1]  Yonina C. Eldar,et al.  Introduction to Compressed Sensing , 2022 .

[2]  Yoram Bresler,et al.  Globally convergent edge-preserving regularized reconstruction: an application to limited-angle tomography , 1998, IEEE Trans. Image Process..

[3]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[4]  Hédy Attouch,et al.  Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Lojasiewicz Inequality , 2008, Math. Oper. Res..

[5]  Mila Nikolova,et al.  Efficient Reconstruction of Piecewise Constant Images Using Nonsmooth Nonconvex Minimization , 2008, SIAM J. Imaging Sci..

[6]  A. Miele,et al.  Study on a memory gradient method for the minimization of functions , 1969 .

[7]  David L. Donoho,et al.  Neighborly Polytopes And Sparse Solution Of Underdetermined Linear Equations , 2005 .

[8]  K. Lange.  Convergence of EM image reconstruction algorithms with Gibbs smoothing. , 1990, IEEE transactions on medical imaging.

[9]  Mariano Rivera,et al.  Efficient half-quadratic regularization with granularity control , 2003, Image Vis. Comput..

[10]  J. Idier,et al.  On global and local convergence of half-quadratic algorithms , 2002, Proceedings. International Conference on Image Processing.

[11]  Michael Elad,et al.  Analysis versus synthesis in signal priors , 2006, 2006 14th European Signal Processing Conference.

[12]  V. K. Govindan,et al.  Nonlocal-Means Image Denoising Technique Using Robust M-Estimator , 2010, Journal of Computer Science and Technology.

[13]  Hédy Attouch,et al.  On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , 2008, Math. Program..

[14]  Curtis R. Vogel,et al.  Iterative Methods for Total Variation Denoising , 1996, SIAM J. Sci. Comput..

[15]  R. Tyrrell Rockafellar,et al.  Variational Analysis , 1998, Grundlehren der mathematischen Wissenschaften.

[16]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Mila Nikolova,et al.  Analysis of the Recovery of Edges in Images and Signals by Minimizing Nonconvex Regularized Least-Squares , 2005, Multiscale Model. Simul..

[18]  D. Titterington.  General structure of regularization procedures in image reconstruction , 1985 .

[19]  B Muralidhara,et al.  A Majorize - Minimize Strategy for Subspace Optimization Applied to Image Restoration , 2014 .

[20]  Pedro F. Felzenszwalb,et al.  Efficient belief propagation for early vision , 2004, CVPR 2004.

[21]  Michel Barlaud,et al.  Deterministic edge-preserving regularization in computed imaging , 1997, IEEE Trans. Image Process..

[22]  Patrick L. Combettes,et al.  Alternating proximal algorithm for blind image recovery , 2010, 2010 IEEE International Conference on Image Processing.

[23]  C. Jutten,et al.  SRF: Matrix completion based on smoothed rank function , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[24]  BolteJérôme,et al.  Proximal Alternating Minimization and Projection Methods for Nonconvex Problems , 2010 .

[25]  Michael Elad,et al.  L1-L2 Optimization in Signal and Image Processing , 2010, IEEE Signal Processing Magazine.

[26]  Nick G. Kingsbury,et al.  Restoration of images and 3D data to higher resolution by deconvolution with sparsity regularization , 2010, 2010 IEEE International Conference on Image Processing.

[27]  Richard M. Leahy,et al.  Statistic-based MAP image-reconstruction from Poisson data using Gibbs priors , 1992, IEEE Trans. Signal Process..

[28]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[29]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[30]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[31]  J. Idier,et al.  Convergence of Conjugate Gradient Methods with a Closed-Form Stepsize Formula , 2008 .

[32]  R. Zabih,et al.  Bayesian parallel imaging with edge‐preserving priors , 2007, Magnetic resonance in medicine.

[33]  Hugues Talbot,et al.  A Fast Solver for Truncated-Convex Priors: Quantized-Convex Split Moves , 2011, EMMCVPR.

[34]  Ashish Raj,et al.  A graph cut algorithm for generalized image deconvolution , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[35]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[36]  Christian Jutten,et al.  A Fast Approach for Overcomplete Sparse Decomposition Based on Smoothed $\ell ^{0}$ Norm , 2008, IEEE Transactions on Signal Processing.

[37]  Hugues Talbot,et al.  A Memory Gradient algorithm for ℓ2 — ℓ0 regularization with applications to image restoration , 2011, 2011 18th IEEE International Conference on Image Processing.

[38]  J. Dennis,et al.  Techniques for nonlinear least squares and robust regression , 1978 .

[39]  Kaushik Mahata,et al.  An approximate L0 norm minimization algorithm for compressed sensing , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[40]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[41]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[42]  Michael Elad,et al.  Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization , 2007 .

[43]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Patrick Siarry,et al.  Optimization in signal and image processing , 2009 .

[45]  Michael Unser,et al.  Hessian-Based Norm Regularization for Image Restoration With Biomedical Applications , 2012, IEEE Transactions on Image Processing.

[46]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[47]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[48]  K. Kurdyka,et al.  Wf-stratification of subanalytic functions and the Lojasiewicz inequality , 1994 .

[49]  Yonina C. Eldar,et al.  Block-Sparse Signals: Uncertainty Relations and Efficient Recovery , 2009, IEEE Transactions on Signal Processing.

[50]  Adrian S. Lewis,et al.  Clarke Subgradients of Stratifiable Functions , 2006, SIAM J. Optim..

[51]  J. Pesquet,et al.  Wavelet thresholding for some classes of non–Gaussian noise , 2002 .

[52]  R. Zabih,et al.  Efficient Graph-Based Energy Minimization Methods in Computer Vision , 1999 .

[53]  Guillermo Sapiro,et al.  Robust anisotropic diffusion , 1998, IEEE Trans. Image Process..

[54]  Olga Veksler.  Graph Cut Based Optimization for MRFs with Truncated Convex Priors , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  P B Hoffer,et al.  Computerized three-dimensional segmented human anatomy. , 1994, Medical physics.

[56]  M. Teboulle,et al.  A smoothing technique for nondifferentiable optimization problems , 1988 .

[57]  Adrian S. Lewis,et al.  The [barred L]ojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems , 2006, SIAM J. Optim..

[58]  Azriel Rosenfeld,et al.  Robust regression methods for computer vision: A review , 1991, International Journal of Computer Vision.

[59]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[60]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[61]  W. Hager,et al.  A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS , 2005 .

[62]  J. W. Cantrell.  Relation between the memory gradient method and the Fletcher-Reeves method , 1969 .