A Proximal Strictly Contractive Peaceman-Rachford Splitting Method for Convex Programming with Applications to Imaging

A strictly contractive Peaceman--Rachford splitting method was proposed recently for solving separable convex programming problems. In this paper we further discuss a proximal version of this method, where a subproblem at each iteration is regularized by a proximal point term. The resulting regularized subproblem thus may have closed-form or easily computable solutions, especially in some interesting applications such as a class of sparse and low-rank optimization models. We establish the worst-case convergence rate measured by the iteration complexity in both the ergodic and nonergodic senses for the new algorithm. Some applications arising in image processing are tested to demonstrate the efficiency of the new algorithm.

[1]  S. Deans.  The Radon Transform and Some of Its Applications , 1983 .

[2]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[3]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[4]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[5]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[6]  Zhi-Quan Luo,et al.  A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization , 2012, SIAM J. Optim..

[7]  José M. Bioucas-Dias,et al.  A fast algorithm for the constrained formulation of compressive image reconstruction and other linear inverse problems , 2009, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[8]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[9]  Jacques Froment,et al.  Constrained Total Variation Minimization and Application in Computerized Tomography , 2005, EMMCVPR.

[10]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[11]  B. V. Dean,et al.  Studies in Linear and Non-Linear Programming. , 1959 .

[12]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[13]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[14]  Xiaoming Yuan,et al.  Linearized alternating direction method of multipliers for sparse group and fused LASSO models , 2014, Comput. Stat. Data Anal..

[15]  Wotao Yin,et al.  Parallel Multi-Block ADMM with o(1 / k) Convergence , 2013, Journal of Scientific Computing.

[16]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[17]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[18]  H. H. Rachford,et al.  The Numerical Solution of Parabolic and Elliptic Differential Equations , 1955 .

[19]  Nikos Komodakis,et al.  Playing with Duality: An overview of recent primal?dual approaches for solving large-scale optimization problems , 2014, IEEE Signal Process. Mag..

[20]  Gregory Beylkin,et al.  Discrete radon transform , 1987, IEEE Trans. Acoust. Speech Signal Process..

[21]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[22]  Michael K. Ng,et al.  Coupled Variational Image Decomposition and Restoration Model for Blurred Cartoon-Plus-Texture Images With Missing Pixels , 2013, IEEE Transactions on Image Processing.

[23]  Avinash C. Kak,et al.  Principles of computerized tomographic imaging , 2001, Classics in applied mathematics.

[24]  Bingsheng He,et al.  A new inexact alternating directions method for monotone variational inequalities , 2002, Math. Program..

[25]  M. J. D. Powell,et al.  A method for nonlinear constraints in minimization problems , 1969 .

[26]  Xinxin Li.  Some operator splitting methods for convex optimization , 2014 .

[27]  Xiaoming Yuan,et al.  A Generalized Proximal Point Algorithm and Its Convergence Rate , 2014, SIAM J. Optim..

[28]  Tony F. Chan,et al.  A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science , 2010, SIAM J. Imaging Sci..

[29]  J. Pesquet,et al.  A Parallel Inertial Proximal Optimization Method , 2012 .

[30]  Michael K. Ng,et al.  Inexact Alternating Direction Methods for Image Recovery , 2011, SIAM J. Sci. Comput..

[31]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[32]  Jonathan Eckstein.  Augmented Lagrangian and Alternating Direction Methods for Convex Optimization: A Tutorial and Some Illustrative Computational Results , 2012 .

[33]  Xiangfeng Wang,et al.  The Linearized Alternating Direction Method of Multipliers for Dantzig Selector , 2012, SIAM J. Sci. Comput..

[34]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[35]  Roland Glowinski,et al.  On Alternating Direction Methods of Multipliers: A Historical Perspective , 2014, Modeling, Simulation and Optimization for Science and Technology.

[36]  Bingsheng He,et al.  A Strictly Contractive Peaceman-Rachford Splitting Method for Convex Programming , 2014, SIAM J. Optim..

[37]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[38]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[39]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[40]  Arkadi Nemirovski,et al.  Prox-Method with Rate of Convergence O(1/t) for Variational Inequalities with Lipschitz Continuous Monotone Operators and Smooth Convex-Concave Saddle Point Problems , 2004, SIAM J. Optim..

[41]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[42]  D. Gabay.  Applications of the method of multipliers to variational inequalities , 1983 .

[43]  Zhi-Quan Luo,et al.  Iteration complexity analysis of block coordinate descent methods , 2013, Mathematical Programming.

[44]  P. Tseng.  Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .

[45]  Raymond H. Chan,et al.  Constrained Total Variation Deblurring Models and Fast Algorithms Based on Alternating Direction Method of Multipliers , 2013, SIAM J. Imaging Sci..

[46]  Bingsheng He,et al.  On the convergence rate of Douglas–Rachford operator splitting method , 2015, Math. Program..

[47]  Xiang Ren,et al.  Linearized Alternating Direction Method with Adaptive Penalty and Warm Starts for Fast Solving Transform Invariant Low-Rank Textures , 2012, International Journal of Computer Vision.

[48]  Emmanuel J. Candès,et al.  NESTA: A Fast and Accurate First-Order Method for Sparse Recovery , 2009, SIAM J. Imaging Sci..

[49]  José Mario Martínez,et al.  Nonmonotone Spectral Projected Gradient Methods on Convex Sets , 1999, SIAM J. Optim..

[50]  K. T. Block,et al.  Undersampled radial MRI with multiple coils. Iterative image reconstruction using a total variation constraint , 2007, Magnetic resonance in medicine.

[51]  Stanley Osher,et al.  A Unified Primal-Dual Algorithm Framework Based on Bregman Iteration , 2010, J. Sci. Comput..

[52]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[53]  Bingsheng He,et al.  On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers , 2014, Numerische Mathematik.

[54]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[55]  Jonathan Eckstein.  Splitting methods for monotone operators with applications to parallel optimization , 1989 .

[56]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[57]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[58]  Michael K. Ng,et al.  Solving Constrained Total-variation Image Restoration and Reconstruction Problems via Alternating Direction Methods , 2010, SIAM J. Sci. Comput..

[59]  Yurii Nesterov,et al.  Gradient methods for minimizing composite functions , 2012, Mathematical Programming.

[60]  J.-C. Pesquet,et al.  A Douglas–Rachford Splitting Approach to Nonsmooth Convex Variational Signal Recovery , 2007, IEEE Journal of Selected Topics in Signal Processing.

[61]  Junfeng Yang,et al.  Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization , 2012, Math. Comput..

[62]  Michael Elad,et al.  Analysis versus synthesis in signal priors , 2006, 2006 14th European Signal Processing Conference.