Distributed and Consensus Optimization for Non-smooth Image Reconstruction

We present a primal-dual algorithm with consensus constraints that can effectively handle large-scale and complex data in a variety of non-smooth image reconstruction problems. In particular, we focus on the case that the data fidelity term can be decomposed into multiple relatively simple functions and deployed to parallel computing units to cooperatively solve for a consensual solution of the original problem. In this case, the subproblems usually have closed form solution or can be solved efficiently at local computing units, and hence the per-iteration computation complexity is very low. A comprehensive convergence analysis of the algorithm, including convergence rate, is established.

[1]  William W. Hager,et al.  Bregman operator splitting with variable stepsize for total variation image reconstruction , 2013, Comput. Optim. Appl..

[2]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[3]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[4]  Jian-Feng Cai,et al.  Split Bregman Methods and Frame Based Image Restoration , 2009, Multiscale Model. Simul..

[5]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[6]  Annie I-An Chen,et al.  Fast Distributed First-Order Methods , 2012 .

[7]  Bingsheng He,et al.  Convergence Analysis of Primal-Dual Algorithms for a Saddle-Point Problem: From Contraction Perspective , 2012, SIAM J. Imaging Sci..

[8]  Xavier Bresson,et al.  Geometric Applications of the Split Bregman Method: Segmentation and Surface Reconstruction , 2010, J. Sci. Comput..

[9]  Stphane Mallat,et al.  A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way , 2008 .

[10]  Arkadi S. Nemirovsky,et al.  Information-based complexity of linear operator equations , 1992, J. Complex..

[11]  Tony F. Chan,et al.  A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science , 2010, SIAM J. Imaging Sci..

[12]  P. Bühlmann,et al.  The group lasso for logistic regression , 2008 .

[13]  Gene H. Golub,et al.  A Nonlinear Primal-Dual Method for Total Variation-Based Image Restoration , 1999, SIAM J. Sci. Comput..

[14]  Yunmei Chen,et al.  Optimal Primal-Dual Methods for a Class of Saddle Point Problems , 2013, SIAM J. Optim..

[15]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[16]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[17]  B. V. Dean,et al.  Studies in Linear and Non-Linear Programming. , 1959 .

[18]  B. V. Dean,et al.  Studies in Linear and Non-Linear Programming. , 1959 .

[19]  Michael B. Wakin.  Sparse Image and Signal Processing: Wavelets, Curvelets, Morphological Diversity (Starck, J.-L., et al; 2010) [Book Reviews] , 2011, IEEE Signal Processing Magazine.

[20]  Valeria Ruggiero,et al.  On the Convergence of Primal–Dual Hybrid Gradient Algorithms for Total Variation Image Restoration , 2012, Journal of Mathematical Imaging and Vision.

[21]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[22]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[23]  Asuman E. Ozdaglar,et al.  Constrained Consensus and Optimization in Multi-Agent Networks , 2008, IEEE Transactions on Automatic Control.

[24]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[25]  Renato D. C. Monteiro,et al.  Iteration-Complexity of Block-Decomposition Algorithms and the Alternating Direction Method of Multipliers , 2013, SIAM J. Optim..

[26]  Mingqiang Zhu,et al.  An Efficient Primal-Dual Hybrid Gradient Algorithm For Total Variation Image Restoration , 2008 .

[27]  Karl Henrik Johansson,et al.  On decentralized negotiation of optimal consensus , 2008, Autom..

[28]  José M. F. Moura,et al.  Fast Distributed Gradient Methods , 2011, IEEE Transactions on Automatic Control.

[29]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[30]  Xiangfeng Wang,et al.  Multi-Agent Distributed Optimization via Inexact Consensus ADMM , 2014, IEEE Transactions on Signal Processing.

[31]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[32]  Yunmei Chen,et al.  Computational Acceleration for MR Image Reconstruction in Partially Parallel Imaging , 2011, IEEE Transactions on Medical Imaging.

[33]  Ernie Esser,et al.  Applications of Lagrangian-Based Alternating Direction Methods and Connections to Split Bregman , 2009 .

[34]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[35]  Anand D. Sarwate,et al.  Broadcast Gossip Algorithms for Consensus , 2009, IEEE Transactions on Signal Processing.

[36]  Stephen J. Wright,et al.  Duality-based algorithms for total-variation-regularized image restoration , 2010, Comput. Optim. Appl..

[37]  J. Moreau.  Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[38]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[39]  R. Tibshirani,et al.  A note on the group lasso and a sparse group lasso , 2010, 1001.0736.

[40]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing - The Sparse Way, 3rd Edition , 2008 .

[41]  Yurii Nesterov,et al.  Primal-dual subgradient methods for convex problems , 2005, Math. Program..

[42]  Stanley Osher,et al.  A Unified Primal-Dual Algorithm Framework Based on Bregman Iteration , 2010, J. Sci. Comput..

[43]  Xue-Cheng Tai,et al.  Augmented Lagrangian Method, Dual Methods, and Split Bregman Iteration for ROF, Vectorial TV, and High Order Models , 2010, SIAM J. Imaging Sci..

[44]  Qing Ling,et al.  On the Linear Convergence of the ADMM in Decentralized Consensus Optimization , 2013, IEEE Transactions on Signal Processing.

[45]  F ChanTony,et al.  A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science , 2010 .

[46]  Michael Hintermüller,et al.  An Infeasible Primal-Dual Algorithm for Total Bounded Variation-Based Inf-Convolution-Type Image Restoration , 2006, SIAM J. Sci. Comput..

[47]  Jean-Philippe Vert,et al.  Group lasso with overlap and graph lasso , 2009, ICML '09.

[48]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[49]  Y. Nesterov.  A method for unconstrained convex minimization problem with the rate of convergence o(1/k^2) , 1983 .

[50]  Yurii Nesterov.  Nonsmooth Convex Optimization , 2004 .