Epigraphical splitting for solving constrained convex formulations of inverse problems with proximal tools

We propose a proximal approach to deal with a class of convex variational problems involving nonlinear constraints. A large family of constraints, proven to be effective in the solution of inverse problems, can be expressed as the lower level set of a sum of convex functions evaluated over different, but possibly overlapping, blocks of the signal. For such constraints, the associated projection operator generally does not have a simple form. We circumvent this difficulty by splitting the lower level set into as many epigraphs as functions involved in the sum. A closed half-space constraint is also enforced, in order to limit the sum of the introduced epigraphical variables to the upper bound of the original lower level set. In this paper, we focus on a family of constraints involving linear transforms of distance functions to a convex set or $\ell_{1,p}$ norms with $p\in \{1,2,\infty\}$. In these cases, the projection onto the epigraph of the involved function has a closed form expression. The proposed approach is validated in the context of image restoration with missing samples, by making use of constraints based on Non-Local Total Variation. Experiments show that our method leads to significant improvements in term of convergence speed over existing algorithms for solving similar constrained problems. A second application to a pulse shape design problem is provided in order to illustrate the flexibility of the proposed approach.

[1]  Patrick L. Combettes,et al.  Proximal Thresholding Algorithm for Minimization over Orthonormal Bases , 2007, SIAM J. Optim..

[2]  Patrick L. Combettes,et al.  Convex set theoretic image recovery by extrapolated iterations of parallel subgradient projections , 1997, IEEE Trans. Image Process..

[3]  Mohamed-Jalal Fadili,et al.  A Proximal Iteration for Deconvolving Poisson Noisy Images Using Sparse Representations , 2008, IEEE Transactions on Image Processing.

[4]  Nelly Pustelnik,et al.  A Nonlocal Structure Tensor-Based Approach for Multicomponent Image Recovery Problems , 2014, IEEE Transactions on Image Processing.

[5]  Charles Kervrann,et al.  Local Adaptivity to Variable Smoothness for Exemplar-Based Image Regularization and Representation , 2008, International Journal of Computer Vision.

[6]  H. Upmeier ANALYSIS ON SYMMETRIC CONES (Oxford Mathematical Monographs) , 1996 .

[7]  Fang Liu,et al.  Multivariate Compressive Sensing for Image Reconstruction in the Wavelet Domain: Using Scale Mixture Models , 2011, IEEE Transactions on Image Processing.

[8]  Tony F. Chan,et al.  Structure-Texture Image Decomposition—Modeling, Algorithms, and Parameter Selection , 2006, International Journal of Computer Vision.

[9]  Gabriel Peyré,et al.  A Review of Adaptive Image Representations , 2011, IEEE Journal of Selected Topics in Signal Processing.

[10]  Sergios Theodoridis,et al.  Adaptive Learning in Complex Reproducing Kernel Hilbert Spaces Employing Wirtinger's Subgradients , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[11]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[12]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[13]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[14]  Mohamed-Jalal Fadili,et al.  Group sparsity with overlapping partition functions , 2011, 2011 19th European Signal Processing Conference.

[15]  Klaas Paul Pruessmann,et al.  A Fast Wavelet-Based Reconstruction Method for Magnetic Resonance Imaging , 2011, IEEE Transactions on Medical Imaging.

[16]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[17]  Jean-François Aujol,et al.  Some First-Order Algorithms for Total Variation Based Image Restoration , 2009, Journal of Mathematical Imaging and Vision.

[18]  A. N. Tikhonov,et al.  REGULARIZATION OF INCORRECTLY POSED PROBLEMS , 1963 .

[19]  Thierry Blu,et al.  Monte-Carlo Sure: A Black-Box Optimization of Regularization Parameters for General Denoising Algorithms , 2008, IEEE Transactions on Image Processing.

[20]  Stephen J. Wright Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[21]  R. Chan,et al.  Minimization and parameter estimation for seminorm regularization models with I-divergence constraints , 2013 .

[22]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[23]  Hugues Talbot,et al.  A primal-dual proximal splitting approach for restoring data corrupted with poisson-gaussian noise , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[24]  Joumana Farah,et al.  A Parallel Proximal Splitting Method for Disparity Estimation from Multicomponent Images Under Illumination Variation , 2012, Journal of Mathematical Imaging and Vision.

[25]  S. Osher,et al.  Image restoration: Total variation, wavelet frames, and beyond , 2012 .

[26]  Amel Benazza-Benyahia,et al.  A hierarchical Bayesian model for frame representation , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[27]  Xiaoqun Zhang,et al.  A primal–dual fixed point algorithm for convex separable minimization with applications to image restoration , 2013 .

[28]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[29]  Laurent D. Cohen,et al.  Non-local Regularization of Inverse Problems , 2008, ECCV.

[30]  Jonathan Eckstein Parallel alternating direction multiplier decomposition of convex programs , 1994 .

[31]  M. Sezan,et al.  Image Restoration by the Method of Convex Projections: Part 2-Applications and Numerical Results , 1982, IEEE Transactions on Medical Imaging.

[32]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[33]  Patrick L. Combettes,et al.  Proximal Algorithms for Multicomponent Image Recovery Problems , 2011, Journal of Mathematical Imaging and Vision.

[34]  Rémi Gribonval,et al.  Beyond the Narrowband Approximation: Wideband Convex Methods for Under-Determined Reverberant Audio Source Separation , 2010, IEEE Transactions on Audio, Speech, and Language Processing.

[35]  A. Bovik F Mean Squared Error: Love It or Leave It? , 2009 .

[36]  Antonin Chambolle,et al.  Image Decomposition into a Bounded Variation Component and an Oscillating Component , 2005, Journal of Mathematical Imaging and Vision.

[37]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[38]  Carola-Bibiane Schönlieb,et al.  Subspace Correction Methods for Total Variation and 1-Minimization , 2007, SIAM J. Numer. Anal..

[39]  Volkan Cevher,et al.  Filtered Variation method for denoising and sparse signal processing , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  Guy Gilboa,et al.  Nonlocal Linear Image Regularization and Supervised Segmentation , 2007, Multiscale Model. Simul..

[41]  Patrick L. Combettes,et al.  On the effectiveness of projection methods for convex feasibility problems with linear inequality constraints , 2009, Computational Optimization and Applications.

[42]  H. Trussell,et al.  The feasible solution in signal restoration , 1984 .

[43]  Aleksandra Pizurica,et al.  Estimating the probability of the presence of a signal of interest in multiresolution single- and multiband image denoising , 2006, IEEE Transactions on Image Processing.

[44]  Nelly Pustelnik,et al.  Relaxing Tight Frame Condition in Parallel Proximal Methods for Signal Restoration , 2011, IEEE Transactions on Signal Processing.

[45]  Gabriele Steidl,et al.  Removing Multiplicative Noise by Douglas-Rachford Splitting Methods , 2010, Journal of Mathematical Imaging and Vision.

[46]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[47]  Ömer Deniz Akyildiz,et al.  An analysis prior based decomposition method for audio signals , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[48]  Gabriele Steidl,et al.  Deblurring Poissonian images by split Bregman techniques , 2010, J. Vis. Commun. Image Represent..

[49]  Valérie R. Wajs,et al.  A variational formulation for frame-based inverse problems , 2007 .

[50]  J.-C. Pesquet,et al.  A Douglas–Rachford Splitting Approach to Nonsmooth Convex Variational Signal Recovery , 2007, IEEE Journal of Selected Topics in Signal Processing.

[51]  Donald Goldfarb,et al.  Second-order cone programming , 2003, Math. Program..

[52]  Xavier Bresson,et al.  Bregmanized Nonlocal Regularization for Deconvolution and Sparse Reconstruction , 2010, SIAM J. Imaging Sci..

[53]  Camille Couprie,et al.  Dual constrained TV-based regularization , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[54]  Giacomo Boracchi,et al.  Foveated self-similarity in nonlocal image filtering , 2012, Electronic Imaging.

[55]  Gilles Aubert,et al.  Efficient Schemes for Total Variation Minimization Under Constraints in Image Processing , 2009, SIAM J. Sci. Comput..

[56]  Kim-Chuan Toh,et al.  An introduction to a class of matrix cone programming , 2012, Mathematical Programming.

[57]  Martin Burger,et al.  The iteratively regularized Gauss–Newton method with convex constraints and applications in 4Pi microscopy , 2011, 1106.5812.

[58]  Defeng Sun,et al.  Semismooth Homeomorphisms and Strong Stability of Semidefinite and Lorentz Complementarity Problems , 2003, Math. Oper. Res..

[59]  Nelly Pustelnik,et al.  Epigraphical proximal projection for sparse multiclass SVM , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[60]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[61]  J. Moreau Proximité et dualité dans un espace hilbertien , 1965 .

[62]  R. Rockafellar Monotone Operators and the Proximal Point Algorithm , 1976 .

[63]  Shunsuke Ono,et al.  Poisson image restoration with likelihood constraint via hybrid steepest descent method , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[64]  B. Martinet,et al.  R'egularisation d''in'equations variationnelles par approximations successives , 1970 .

[65]  Tony F. Chan,et al.  A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science , 2010, SIAM J. Imaging Sci..

[66]  P. L. Combettes The foundations of set theoretic estimation , 1993 .

[67]  Amel Benazza-Benyahia,et al.  Satellite image restoration in the context of a spatially varying point spread function. , 2010, Journal of the Optical Society of America. A, Optics, image science, and vision.

[68]  Patrick L. Combettes,et al.  A block-iterative surrogate constraint splitting method for quadratic signal recovery , 2003, IEEE Trans. Signal Process..

[69]  Boris Polyak Minimization of unsmooth functionals , 1969 .

[70]  Guy Gilboa,et al.  Nonlocal Operators with Applications to Image Processing , 2008, Multiscale Model. Simul..

[71]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[72]  Patrick L. Combettes,et al.  A Monotone+Skew Splitting Model for Composite Monotone Inclusions in Duality , 2010, SIAM J. Optim..

[73]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[74]  S. Mallat A wavelet tour of signal processing , 1998 .

[75]  M. Bingham Analysis on Symmetric Cones (oxford Mathematical Monographs) , 2006 .

[76]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[77]  Patrick L. Combettes,et al.  Inconsistent signal feasibility problems: least-squares solutions in a product space , 1994, IEEE Trans. Signal Process..

[78]  José M. Bioucas-Dias,et al.  Restoration of Poissonian Images Using Alternating Direction Optimization , 2010, IEEE Transactions on Image Processing.

[79]  Stephen J. Wright,et al.  Simultaneous Variable Selection , 2005, Technometrics.

[80]  I. Yamada The Hybrid Steepest Descent Method for the Variational Inequality Problem over the Intersection of Fixed Point Sets of Nonexpansive Mappings , 2001 .

[81]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[82]  P. L. Combettes,et al.  A proximal decomposition method for solving convex variational inverse problems , 2008, 0807.2617.

[83]  Émilie Chouzenoux,et al.  A Majorize–Minimize Strategy for Subspace Optimization Applied to Image Restoration , 2011, IEEE Transactions on Image Processing.

[84]  P. Tseng Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization , 2001 .

[85]  Laurent Jacques,et al.  A panorama on multiscale geometric representations, intertwining spatial, directional and frequency selectivity , 2011, Signal Process..

[86]  Julien Mairal,et al.  Optimization with Sparsity-Inducing Penalties , 2011, Found. Trends Mach. Learn..

[87]  Jean-Michel Morel,et al.  Secrets of image denoising cuisine* , 2012, Acta Numerica.

[88]  J. Pesquet,et al.  A Parallel Inertial Proximal Optimization Method , 2012 .

[89]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[90]  Trevor Darrell,et al.  An efficient projection for l1, ∞ regularization , 2009, ICML '09.

[91]  D. Youla,et al.  Image Restoration by the Method of Convex Projections: Part 1ߞTheory , 1982, IEEE Transactions on Medical Imaging.

[92]  M. Teboulle,et al.  A smoothing technique for nondifferentiable optimization problems , 1988 .

[93]  Mohamed-Jalal Fadili,et al.  Total Variation Projection With First Order Schemes , 2011, IEEE Transactions on Image Processing.

[94]  Bang Công Vu,et al.  A splitting algorithm for dual monotone inclusions involving cocoercive operators , 2011, Advances in Computational Mathematics.

[95]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[96]  M. Reha Civanlar,et al.  Optimal pulse shape design for digital communication systems by projections onto convex sets , 1995, IEEE Transactions on Communications.

[97]  Giovanni Chierchia,et al.  Parallel implementations of a disparity estimation algorithm based on a Proximal splitting method , 2012, 2012 Visual Communications and Image Processing.

[98]  Nikolas P. Galatsanos,et al.  Methods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation , 1992, IEEE Trans. Image Process..

[99]  J. Pesquet,et al.  Parallel algorithm and hybrid regularization for dynamic PET reconstruction , 2010, IEEE Nuclear Science Symposuim & Medical Imaging Conference.

[100]  Ilker Bayram,et al.  Directional Total Variation , 2012, IEEE Signal Processing Letters.

[101]  Gabriele Steidl,et al.  Homogeneous Penalizers and Constraints in Convex Image Restoration , 2013, Journal of Mathematical Imaging and Vision.

[102]  Amel Benazza-Benyahia,et al.  A wavelet-based regularized reconstruction algorithm for SENSE parallel MRI with applications to neuroimaging , 2011, Medical Image Anal..

[103]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[104]  Dianne P. O'Leary,et al.  The Use of the L-Curve in the Regularization of Discrete Ill-Posed Problems , 1993, SIAM J. Sci. Comput..

[105]  Aleksandra Pizurica,et al.  Split-Bregman-based sparse-view CT reconstruction , 2011 .

[106]  José M. Bioucas-Dias,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[107]  Patrick L. Combettes,et al.  Hard-constrained inconsistent signal feasibility problems , 1999, IEEE Trans. Signal Process..

[108]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[109]  J. Aujol,et al.  Some algorithms for total variation based image restoration , 2008 .

[110]  I. Yamada,et al.  The Adaptive Projected Subgradient Method over the Fixed Point Set of Strongly Attracting Nonexpansive Mappings , 2006 .

[111]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[112]  Sergios Theodoridis,et al.  Adaptive Learning in a World of Projections , 2011, IEEE Signal Processing Magazine.

[113]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[114]  Marc Teboulle,et al.  A proximal-based decomposition method for convex minimization problems , 1994, Math. Program..