A Fast Stochastic Plug-and-Play ADMM for Imaging Inverse Problems

In this work we propose an efficient stochastic plug-and-play (PnP) algorithm for imaging inverse problems. The PnP stochastic gradient descent methods have been recently proposed and shown improved performance in some imaging applications over standard deterministic PnP methods. However, current stochastic PnP methods need to frequently compute the image denoisers which can be computationally expensive. To overcome this limitation, we propose a new stochastic PnP-ADMM method which is based on introducing stochastic gradient descent inner-loops within an inexact ADMM framework. We provide the theoretical guarantee on the fixed-point convergence for our algorithm under standard assumptions. Our numerical results demonstrate the effectiveness of our approach compared with state-of-the-art PnP methods.

[1]  Per Christian Hansen,et al.  AIR Tools - A MATLAB package of algebraic iterative reconstruction methods , 2012, J. Comput. Appl. Math..

[2]  Brendt Wohlberg,et al.  An Online Plug-and-Play Algorithm for Regularized Image Reconstruction , 2018, IEEE Transactions on Computational Imaging.

[3]  Hassan Mansour,et al.  A Plug-and-Play Priors Approach for Solving Nonlinear Imaging Inverse Problems , 2017, IEEE Signal Processing Letters.

[4]  Mohamed-Jalal Fadili,et al.  Local Convergence Properties of Douglas–Rachford and Alternating Direction Method of Multipliers , 2017, Journal of Optimization Theory and Applications.

[5]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[6]  Adrian S. Lewis,et al.  The [barred L]ojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems , 2006, SIAM J. Optim..

[7]  Philip Schniter,et al.  Regularization by Denoising: Clarifications and New Interpretations , 2018, IEEE Transactions on Computational Imaging.

[8]  Radu Ioan Bot,et al.  Inertial Douglas-Rachford splitting for monotone inclusion problems , 2014, Appl. Math. Comput..

[9]  A. Chambolle,et al.  On the Convergence of the Iterates of the “Fast Iterative Shrinkage/Thresholding Algorithm” , 2015, J. Optim. Theory Appl..

[10]  Yunjin Chen,et al.  Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[12]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[13]  Xiaohan Chen,et al.  Plug-and-Play Methods Provably Converge with Properly Trained Denoisers , 2019, ICML.

[14]  Mike E. Davies,et al.  Gradient Projection Iterative Sketch for Large-Scale Constrained Least-Squares , 2016, ICML.

[15]  L. Rosasco,et al.  Convergence of Stochastic Proximal Gradient Algorithm , 2014, Applied Mathematics & Optimization.

[16]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[17]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[18]  Deanna Needell,et al.  Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm , 2013, Mathematical Programming.

[19]  Yoram Singer,et al.  Pegasos: primal estimated sub-gradient solver for SVM , 2011, Math. Program..

[20]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[21]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[22]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[23]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[24]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[25]  Brendt Wohlberg,et al.  Plug-and-Play priors for model based reconstruction , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[26]  Pawel Markiewicz,et al.  Faster PET reconstruction with a stochastic primal-dual hybrid gradient method , 2017, Optical Engineering + Applications.

[27]  Karen O. Egiazarian,et al.  Compressed Sensing Image Reconstruction Via Recursive Spatially Adaptive Filtering , 2007, ICIP.

[28]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[29]  Jing Wang,et al.  Penalized weighted least-squares approach to sinogram noise reduction and image reconstruction for low-dose X-ray computed tomography , 2006, IEEE Transactions on Medical Imaging.

[30]  J. Pesquet,et al.  A Class of Randomized Primal-Dual Algorithms for Distributed Optimization , 2014, 1406.6404.

[31]  Mike Davies,et al.  The Practicality of Stochastic Optimization in Imaging Inverse Problems , 2020, IEEE Transactions on Computational Imaging.

[32]  Benjamin Recht,et al.  Sharp Time–Data Tradeoffs for Linear Inverse Problems , 2015, IEEE Transactions on Information Theory.

[33]  Antonin Chambolle,et al.  Stochastic Primal-Dual Hybrid Gradient Algorithm with Arbitrary Sampling and Imaging Applications , 2017, SIAM J. Optim..

[34]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[35]  Antonin Chambolle,et al.  An introduction to continuous optimization for imaging , 2016, Acta Numerica.

[36]  Peyman Milanfar,et al.  Global Image Denoising , 2014, IEEE Transactions on Image Processing.

[37]  Émilie Chouzenoux,et al.  A Stochastic Majorize-Minimize Subspace Algorithm for Online Penalized Least Squares Estimation , 2015, IEEE Transactions on Signal Processing.

[38]  Mike E. Davies,et al.  Rest-Katyusha: Exploiting the Solution's Structure via Scheduled Restart Schemes , 2018, NeurIPS.

[39]  Karen O. Egiazarian,et al.  Image restoration by sparse 3D transform-domain collaborative filtering , 2008, Electronic Imaging.

[40]  Brendt Wohlberg,et al.  Boosting the Performance of Plug-and-Play Priors via Denoiser Scaling , 2020, 2020 54th Asilomar Conference on Signals, Systems, and Computers.

[41]  Michael Elad,et al.  The Little Engine That Could: Regularization by Denoising (RED) , 2016, SIAM J. Imaging Sci..