A parallel block-coordinate approach for primal-dual splitting with arbitrary random block selection

The solution of many applied problems relies on finding the minimizer of a sum of smooth and/or nonsmooth convex functions possibly involving linear operators. In the last years, primal-dual methods have shown their efficiency to solve such minimization problems, their main advantage being their ability to deal with linear operators with no need to invert them. However, when the problem size becomes increasingly large, the implementation of these algorithms can be complicated, due to memory limitation issues. A simple way to overcome this difficulty consists of splitting the original numerous variables into blocks of smaller dimension, corresponding to the available memory, and to process them separately. In this paper we propose a random block-coordinate primal-dual algorithm, converging almost surely to a solution to the considered minimization problem. Moreover, an application to large-size 3D mesh denoising is provided to show the numerical efficiency of our method.

[1]  Gene H. Golub,et al.  Optimal Surface Smoothing as Filter Design , 1996, ECCV.

[2]  Antonin Chambolle,et al.  Diagonal preconditioning for first order primal-dual algorithms in convex optimization , 2011, 2011 International Conference on Computer Vision.

[3]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[4]  Thomas Hofmann,et al.  Communication-Efficient Distributed Dual Coordinate Ascent , 2014, NIPS.

[5]  Pascal Bianchi,et al.  A stochastic coordinate descent primal-dual algorithm and applications , 2014, 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP).

[6]  Peter Richtárik,et al.  Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function , 2011, Mathematical Programming.

[7]  Patrick L. Combettes,et al.  Stochastic Quasi-Fejér Block-Coordinate Fixed Point Iterations with Random Sweeping , 2014 .

[8]  Émilie Chouzenoux,et al.  A random block-coordinate primal-dual proximal algorithm with application to 3D mesh denoising , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[9]  Xiaoqun Zhang,et al.  A primal–dual fixed point algorithm for convex separable minimization with applications to image restoration , 2013 .

[10]  Abderrahim Elmoataz,et al.  Nonlocal Discrete Regularization on Weighted Graphs: A Framework for Image and Manifold Processing , 2008, IEEE Transactions on Image Processing.

[11]  J. Pesquet,et al.  A Parallel Inertial Proximal Optimization Method , 2012 .

[12]  J. Pesquet,et al.  A Class of Randomized Primal-Dual Algorithms for Distributed Optimization , 2014, 1406.6404.

[13]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[14]  P. L. Combettes,et al.  Variable metric forward–backward splitting with applications to monotone inclusions in duality , 2012, 1206.6791.

[15]  Bang Công Vu,et al.  A splitting algorithm for dual monotone inclusions involving cocoercive operators , 2011, Advances in Computational Mathematics.

[16]  Nikos Komodakis,et al.  Playing with Duality: An overview of recent primal?dual approaches for solving large-scale optimization problems , 2014, IEEE Signal Process. Mag..

[17]  Leo Grady,et al.  Discrete Calculus - Applied Analysis on Graphs for Computational Science , 2010 .

[18]  Gabriele Steidl,et al.  Deblurring Poissonian images by split Bregman techniques , 2010, J. Vis. Commun. Image Represent..

[19]  P. L. Combettes The foundations of set theoretic estimation , 1993 .

[20]  Camille Couprie,et al.  Dual constrained TV-based regularization , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[21]  I. Loris,et al.  On a generalization of the iterative soft-thresholding algorithm for the case of non-separable penalty , 2011, 1104.1087.

[22]  Ieee Staff 2017 25th European Signal Processing Conference (EUSIPCO) , 2017 .

[23]  Émilie Chouzenoux,et al.  A penalized weighted least squares approach for restoring data corrupted with signal-dependent noise , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[24]  Patrick L. Combettes,et al.  A forward-backward view of some primal-dual optimization methods in image recovery , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[25]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[26]  José M. Bioucas-Dias,et al.  Restoration of Poissonian Images Using Alternating Direction Optimization , 2010, IEEE Transactions on Image Processing.

[27]  Émilie Chouzenoux,et al.  A block coordinate variable metric forward–backward algorithm , 2016, Journal of Global Optimization.

[28]  Patrick L. Combettes,et al.  A block-iterative surrogate constraint splitting method for quadratic signal recovery , 2003, IEEE Trans. Signal Process..

[29]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.