A fast dual proximal gradient algorithm for convex minimization and applications

Abstract We consider the convex composite problem of minimizing the sum of a strongly convex function and a general extended valued convex function. We present a dual-based proximal gradient scheme for solving this problem. We show that although the rate of convergence of the dual objective function sequence converges to the optimal value with the rate O ( 1 / k 2 ) , the rate of convergence of the primal sequence is of the order O ( 1 / k ) .

[1]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[2]  H. Uzawa,et al.  Preference, production, and capital: Iterative methods for concave programming , 1989 .

[3]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[4]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[5]  Gilles Aubert,et al.  Efficient Schemes for Total Variation Minimization Under Constraints in Image Processing , 2009, SIAM J. Sci. Comput..

[6]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[7]  Gregory B. Passty.  Ergodic convergence to a zero of the sum of monotone operators in Hilbert space , 1979 .

[8]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[9]  Mohamed-Jalal Fadili,et al.  Total Variation Projection With First Order Schemes , 2011, IEEE Transactions on Image Processing.

[10]  Y. Nesterov.  Gradient methods for minimizing composite objective function , 2007 .

[11]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[12]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[13]  P. Tseng.  Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .

[14]  D. Gabay.  Applications of the method of multipliers to variational inequalities , 1983 .

[15]  P. L. Combettes,et al.  Dualization of Signal Recovery Problems , 2009, 0907.0436.

[16]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[17]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[18]  A. J. Scholl,et al.  NON‐ARCHIMEDEAN ANALYSIS (Grundlehren der mathematischen Wissenschaften 261) , 1986 .

[19]  Marc Teboulle,et al.  Mirror descent and nonlinear projected subgradient methods for convex optimization , 2003, Oper. Res. Lett..

[20]  R. Tyrrell Rockafellar,et al.  Variational Analysis , 1998, Grundlehren der mathematischen Wissenschaften.

[21]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[22]  Marc Teboulle,et al.  Projected subgradient methods with non-Euclidean distances for non-differentiable convex minimization and variational inequalities , 2009, Math. Program..

[23]  R. Glowinski,et al.  Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics , 1987 .