Diagonal preconditioning for first order primal-dual algorithms in convex optimization

In this paper we study preconditioning techniques for the first-order primal-dual algorithm proposed in [5]. In particular, we propose simple and easy to compute diagonal preconditioners for which convergence of the algorithm is guaranteed without the need to compute any step size parameters. As a by-product, we show that for a certain instance of the preconditioning, the proposed algorithm is equivalent to the old and widely unknown alternating step method for monotropic programming [7]. We show numerical results on general linear programming problems and a few standard computer vision problems. In all examples, the preconditioned algorithm significantly outperforms the algorithm of [5].

[1]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[2]  Tony F. Chan,et al.  A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science , 2010, SIAM J. Imaging Sci..

[3]  Xiaoming Yuan,et al.  Convergence analysis of primal-dual algorithms for total variation image restoration , 2010 .

[4]  Christoph Schnörr,et al.  Fast and Exact Primal-Dual Iterations for Variational Problems in Computer Vision , 2010, ECCV.

[5]  Antonin Chambolle,et al.  Total Variation Minimization and a Class of Binary MRF Models , 2005, EMMCVPR.

[6]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[7]  Daniel Cremers,et al.  Curvature regularity for region-based image segmentation and inpainting: A linear programming relaxation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[8]  Eric V. Denardo,et al.  Flows in Networks , 2011 .

[9]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Stanley Osher,et al.  A Unified Primal-Dual Algorithm Framework Based on Bregman Iteration , 2010, J. Sci. Comput..

[11]  Camillo J. Taylor,et al.  Graph Cuts via $\ell_1$ Norm Minimization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Mila Nikolova,et al.  Regularizing Flows for Constrained Matrix-Valued Images , 2004, Journal of Mathematical Imaging and Vision.

[13]  Jonathan Eckstein.  Splitting methods for monotone operators with applications to parallel optimization , 1989 .

[14]  Tony F. Chan,et al.  Aspects of Total Variation Regularized L[sup 1] Function Approximation , 2005, SIAM J. Appl. Math..

[15]  Daniel Cremers,et al.  An algorithm for minimizing the Mumford-Shah functional , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[16]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.