Recent Developments on Primal–Dual Splitting Methods with Applications to Convex Minimization

This chapter presents a survey on primal–dual splitting methods for solving monotone inclusion problems involving maximally monotone operators, linear compositions of parallel sums of maximally monotone operators, and single-valued Lipschitzian or cocoercive monotone operators. The primal–dual algorithms have the remarkable property that the operators involved are evaluated separately in each iteration, either by forward steps in the case of the single-valued ones or by backward steps for the set-valued ones, by using the corresponding resolvents. In the hypothesis that strong monotonicity assumptions for some of the involved operators are fulfilled, accelerated algorithmic schemes are presented and analyzed from the point of view of their convergence. Finally, we discuss the employment of the primal–dual methods in the context of solving convex optimization problems arising in the fields of image denoising and deblurring, support vector machine learning, location theory, portfolio optimization and clustering.

[1]  M. Teboulle,et al.  AN OLD‐NEW CONCEPT OF CONVEX RISK MEASURES: THE OPTIMIZED CERTAINTY EQUIVALENT , 2007 .

[2]  R. Boţ,et al.  Employing different loss functions for the classification of images via supervised learning , 2014 .

[3]  P. Tseng Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .

[4]  C. Zălinescu Convex analysis in general vector spaces , 2002 .

[5]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[6]  Radu Ioan Bot,et al.  Convex risk minimization via proximal splitting methods , 2013, Optim. Lett..

[7]  R. Rockafellar,et al.  On the maximal monotonicity of subdifferential mappings. , 1970 .

[8]  R. Boţ,et al.  Conjugate Duality in Convex Optimization , 2010 .

[9]  P. L. Combettes,et al.  Solving monotone inclusions via compositions of nonexpansive averaged operators , 2004 .

[10]  J. Borwein,et al.  Convex Functions: Constructions, Characterizations and Counterexamples , 2010 .

[11]  Radu Ioan Bot,et al.  A Douglas-Rachford Type Primal-Dual Method for Solving Inclusions with Mixtures of Composite and Parallel-Sum Type Monotone Operators , 2012, SIAM J. Optim..

[12]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[13]  Bernhard Schölkopf,et al.  Combining a Filter Method with SVMs , 2006, Feature Extraction.

[14]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[15]  M. Teboulle,et al.  Expected Utility, Penalty Functions, and Duality in Stochastic Nonlinear Programming , 1986 .

[16]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[17]  Bang Công Vu,et al.  A splitting algorithm for dual monotone inclusions involving cocoercive operators , 2011, Advances in Computational Mathematics.

[18]  H. Attouch A General Duality Principle for the Sum of Two Operators 1 , 1996 .

[19]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[20]  Paul Tseng,et al.  A Modified Forward-backward Splitting Method for Maximal Monotone Mappings 1 , 1998 .

[21]  L. Ljung,et al.  Just Relax and Come Clustering! : A Convexification of k-Means Clustering , 2011 .

[22]  Y. Nesterov A method for unconstrained convex minimization problem with the rate of convergence o(1/k^2) , 1983 .

[23]  Radu Ioan Bot,et al.  Looking for appropriate qualification conditions for subdifferential formulae and dual representations for convex risk measures , 2011, Math. Methods Oper. Res..

[24]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[25]  Radu Ioan Bot,et al.  Optimization problems in statistical learning: Duality and optimality conditions , 2011, Eur. J. Oper. Res..

[26]  S. Simons From Hahn-Banach to monotonicity , 2008 .

[27]  R. Rockafellar Monotone Operators and the Proximal Point Algorithm , 1976 .

[28]  Boris S. Mordukhovich,et al.  Solving a Generalized Heron Problem by Means of Convex Analysis , 2012, Am. Math. Mon..

[29]  B. Mordukhovich,et al.  Applications of variational analysis to a generalized Heron problem , 2011, 1106.0088.

[30]  M. K. Luhandjula Studies in Fuzziness and Soft Computing , 2013 .

[31]  P. L. Combettes Iterative construction of the resolvent of a sum of maximal monotone operators , 2009 .

[32]  Radu Ioan Bot,et al.  Convergence Analysis for a Primal-Dual Monotone + Skew Splitting Algorithm with Applications to Total Variation Minimization , 2012, Journal of Mathematical Imaging and Vision.

[33]  Radu Ioan Bot,et al.  On the convergence rate improvement of a primal-dual splitting algorithm for solving monotone inclusion problems , 2013, Mathematical Programming.

[34]  Patrick L. Combettes,et al.  A Monotone+Skew Splitting Model for Composite Monotone Inclusions in Duality , 2010, SIAM J. Optim..

[35]  Radu Ioan Bot,et al.  A Primal-Dual Splitting Algorithm for Finding Zeros of Sums of Maximal Monotone Operators , 2012, SIAM J. Optim..

[36]  Francis R. Bach,et al.  Clusterpath: an Algorithm for Clustering using Convex Fusion Penalties , 2011, ICML.

[37]  H. Weinert Ekeland, I. / Temam, R., Convex Analysis and Variational Problems. Amsterdam‐Oxford. North‐Holland Publ. Company. 1976. IX, 402 S., Dfl. 85.00. US $ 29.50 (SMAA 1) , 1979 .

[38]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[39]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[40]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[41]  Sorin-Mihai Grad,et al.  Duality in Vector Optimization , 2009, Vector Optimization.

[42]  Eric C. Chi,et al.  Splitting Methods for Convex Clustering , 2013, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[43]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.