Proximal Activation of Smooth Functions in Splitting Algorithms for Convex Image Recovery

Structured convex optimization problems typically involve a mix of smooth and nonsmooth functions. The common practice is to activate the smooth functions via their gradient and the nonsmooth ones via their proximity operator. We show that, although intuitively natural, this approach is not necessarily the most efficient numerically and that, in particular, activating all the functions proximally may be advantageous. To make this viewpoint viable computationally, we derive a number of new examples of proximity operators of smooth convex functions arising in applications. A novel variational model to relax inconsistent convex feasibility problems is also investigated within the proposed framework. Several numerical applications to image recovery are presented to compare the behavior of fully proximal versus mixed proximal/gradient implementations of several splitting algorithms.

[1]  J. Gillis,et al.  Matrix Iterative Analysis , 1961 .

[2]  Loic Landrieu,et al.  Preconditioning of a Generalized Forward-Backward Splitting and Application to Optimization on Graphs , 2015, SIAM J. Imaging Sci..

[3]  Christoph Schnörr,et al.  Unique reconstruction of piecewise-smooth images by minimizing strictly convex nonquadratic functionals , 1994, Journal of Mathematical Imaging and Vision.

[4]  Émilie Chouzenoux,et al.  Dual Block-Coordinate Forward-Backward Algorithm with Application to Deconvolution and Deinterlacing of Video Sequences , 2017, Journal of Mathematical Imaging and Vision.

[5]  Jianhua Ma,et al.  Total Variation-Stokes Strategy for Sparse-View X-ray CT Image Reconstruction , 2014, IEEE Transactions on Medical Imaging.

[6]  Bingsheng He,et al.  Convergence Analysis of Primal-Dual Algorithms for a Saddle-Point Problem: From Contraction Perspective , 2012, SIAM J. Imaging Sci..

[7]  Amel Benazza-Benyahia,et al.  A wavelet-based regularized reconstruction algorithm for SENSE parallel MRI with applications to neuroimaging , 2011, Medical Image Anal..

[8]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[9]  Lionel Moisan,et al.  Posterior Expectation of the Total Variation Model: Properties and Experiments , 2013, SIAM J. Imaging Sci..

[10]  D. Youla,et al.  Extensions of a result on the synthesis of signals in the presence of inconsistent constraints , 1986 .

[11]  Bang Công Vu,et al.  A splitting algorithm for dual monotone inclusions involving cocoercive operators , 2011, Advances in Computational Mathematics.

[12]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[13]  P. L. Combettes,et al.  The Convex Feasibility Problem in Image Recovery , 1996 .

[14]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[15]  Dorian Kermisch,et al.  Image Reconstruction from Phase Information Only , 1970 .

[16]  P. L. Combettes,et al.  Solving monotone inclusions via compositions of nonexpansive averaged operators , 2004 .

[17]  J. Moreau.  Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[18]  Heinz H. Bauschke,et al.  Finding best approximation pairs relative to two closed convex sets in Hilbert spaces , 2004, J. Approx. Theory.

[19]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[20]  C. Byrne.  Iterative Optimization in Inverse Problems , 2014 .

[21]  Patrick L. Combettes,et al.  Hard-constrained inconsistent signal feasibility problems , 1999, IEEE Trans. Signal Process..

[22]  H Stark,et al.  Image restoration by convex projections in the presence of noise. , 1983, Applied optics.

[23]  D. Youla,et al.  Image Restoration by the Method of Convex Projections: Part 1ߞTheory , 1982, IEEE Transactions on Medical Imaging.

[24]  Patrick L. Combettes,et al.  A forward-backward view of some primal-dual optimization methods in image recovery , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[25]  Patrick L. Combettes,et al.  Inconsistent signal feasibility problems: least-squares solutions in a product space , 1994, IEEE Trans. Signal Process..

[26]  Patrick L. Combettes,et al.  Perspective Functions: Properties, Constructions, and Examples , 2016, Set-Valued and Variational Analysis.

[27]  Gabriel Peyré,et al.  Optimal Transport with Proximal Splitting , 2013, SIAM J. Imaging Sci..

[28]  Luis,et al.  Convex Variational Formulation with Smooth Coupling for Multicomponent Signal Decomposition and Recovery , 2009 .

[29]  J. Aubin,et al.  Differential inclusions set-valued maps and viability theory , 1984 .

[30]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[31]  B. R. Hunt,et al.  Digital Image Restoration , 1977 .

[32]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[33]  Yair Censor,et al.  Algorithms and Convergence Results of Projection Methods for Inconsistent Feasibility Problems: A Review , 2018, 1802.07529.

[34]  B. Vu.  A Variable Metric Extension of the Forward–Backward–Forward Algorithm for Monotone Operators , 2012, 1210.2986.

[35]  Patrick L. Combettes,et al.  Consistent learning by composite proximal thresholding , 2015, Mathematical Programming.

[36]  R. Marks,et al.  Signal synthesis in the presence of an inconsistent set of constraints , 1985 .

[37]  Valérie R. Wajs,et al.  A variational formulation for frame-based inverse problems , 2007 .

[38]  J.-C. Pesquet,et al.  A Douglas–Rachford Splitting Approach to Nonsmooth Convex Variational Signal Recovery , 2007, IEEE Journal of Selected Topics in Signal Processing.

[39]  Patrick L. Combettes,et al.  Asynchronous block-iterative primal-dual decomposition methods for monotone inclusions , 2015, Mathematical Programming.

[40]  Stephen L. Keeling,et al.  Total variation based convex filters for medical imaging , 2003, Appl. Math. Comput..

[41]  Aggelos K. Katsaggelos,et al.  Digital image restoration , 2012, IEEE Signal Process. Mag..

[42]  Boris S. Mordukhovich,et al.  Solving a Generalized Heron Problem by Means of Convex Analysis , 2012, Am. Math. Mon..

[43]  Tong Zhang.  Statistical behavior and consistency of classification methods based on convex risk minimization , 2003 .

[44]  PillonettoGianluigi,et al.  Sparse/robust estimation and Kalman smoothing with nonsmooth log-concave densities , 2013 .

[45]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[46]  Patrick L. Combettes,et al.  Solving Coupled Composite Monotone Inclusions by Successive Fejér Approximations of their Kuhn-Tucker Set , 2013, SIAM J. Optim..

[47]  Michael Hintermüller,et al.  An Infeasible Primal-Dual Algorithm for Total Bounded Variation-Based Inf-Convolution-Type Image Restoration , 2006, SIAM J. Sci. Comput..

[48]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[49]  P. L. Combettes,et al.  Variable metric forward–backward splitting with applications to monotone inclusions in duality , 2012, 1206.6791.

[50]  D. Russell Luke,et al.  Local Linear Convergence of the ADMM/Douglas-Rachford Algorithms without Strong Convexity and Application to Statistical Imaging , 2015, SIAM J. Imaging Sci..

[51]  A. Enis Çetin,et al.  Phase and TV Based Convex Sets for Blind Deconvolution of Microscopic Images , 2015, IEEE Journal of Selected Topics in Signal Processing.

[52]  Ingo Steinwart,et al.  Consistency of support vector machines and other regularized kernel classifiers , 2005, IEEE Transactions on Information Theory.

[53]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[54]  Dante C. Youla,et al.  Generalized Image Restoration by the Method of Alternating Orthogonal Projections , 1978 .

[55]  Lieven Vandenberghe,et al.  Primal-Dual Decomposition by Operator Splitting and Applications to Image Deblurring , 2014, SIAM J. Imaging Sci..

[56]  Patrick R. Johnstone,et al.  Projective Splitting with Forward Steps: Asynchronous and Block-Iterative Operator Splitting , 2018, ArXiv.

[57]  Jean-François Aujol,et al.  Stability of Over-Relaxations for the Forward-Backward Algorithm, Application to FISTA , 2015, SIAM J. Optim..

[58]  P. L. Combettes,et al.  Dualization of Signal Recovery Problems , 2009, 0907.0436.

[59]  R. Bellman,et al.  Numerical Inversion of the Laplace Transform: Applications to Biology, Economics Engineering, and Physics , 1967 .

[60]  Michel Defrise,et al.  A note on wavelet-based inversion algorithms , 2002 .

[61]  Y. Censor,et al.  The multiple-sets split feasibility problem and its applications for inverse problems , 2005 .

[62]  Aleksandr Y. Aravkin,et al.  Sparse/robust estimation and Kalman smoothing with nonsmooth log-concave densities: modeling, computation, and theory , 2013, J. Mach. Learn. Res..

[63]  A. Chambolle,et al.  On the Convergence of the Iterates of the “Fast Iterative Shrinkage/Thresholding Algorithm” , 2015, J. Optim. Theory Appl..

[64]  Hugues Talbot,et al.  A Convex Approach for Image Restoration with Exact Poisson-Gaussian Likelihood , 2015, SIAM J. Imaging Sci..

[65]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[66]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.