Perturbed primal-dual dynamics with damping and time scaling coefficients for affine constrained convex optimization problems

We propose a perturbed inertial primal-dual dynamic with damping and scaling coefficients, which involves inertial terms both for primal and dual variables, for a linearly constrained convex optimization problem in a Hilbert setting. With different choices of damping and scaling coefficients, by a Lyapunov analysis approach we discuss the asymptotic properties of the dynamic and prove its fast convergence properties. Our results can be viewed extensions of the existing ones on inertial dynamical systems for the unconstrained convex optimization problem to the linearly constrained convex optimization problem.

[1]  Jean-François Aujol,et al.  Optimal Convergence Rates for Nesterov Acceleration , 2018, SIAM J. Optim..

[2]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[3]  Hao Jiang,et al.  Non-Ergodic Convergence Analysis of Heavy-Ball Algorithms , 2018, AAAI.

[4]  Zaki Chbani,et al.  Fast Convergence of Dynamical ADMM via Time Scaling of Damped Inertial Dynamics , 2021, Journal of Optimization Theory and Applications.

[5]  Fernando Paganini,et al.  Stability of primal-dual gradient dynamics and applications to network optimization , 2010, Autom..

[6]  Michael I. Jordan,et al.  Understanding the acceleration phenomenon via high-resolution differential equations , 2018, Mathematical Programming.

[7]  Na Li,et al.  On the Exponential Stability of Primal-Dual Gradient Dynamics , 2018, IEEE Control Systems Letters.

[8]  Enrique Mallada,et al.  Asymptotic convergence of constrained primal-dual dynamics , 2015, Syst. Control. Lett..

[9]  H. Brezis.  Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert , 1973 .

[10]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[11]  Radu Ioan Bot,et al.  Second Order Forward-Backward Dynamical Systems For Monotone Inclusion Problems , 2015, SIAM J. Control. Optim..

[12]  Juan Peypouquet,et al.  Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity , 2018, Math. Program..

[13]  Hao Luo.  A primal-dual flow for affine constrained convex optimization , 2021 .

[14]  Ya-Ping Fang,et al.  Fast convergence of primal-dual dynamics and algorithms with time scaling for linear equality constrained convex optimization problems , 2021 .

[15]  Hedy Attouch,et al.  Asymptotic stabilization of inertial gradient dynamics with time-dependent viscosity , 2017 .

[16]  J. Bolte,et al.  On damped second-order gradient systems , 2014, 1411.8005.

[17]  Ramzi May,et al.  Asymptotic for the perturbed heavy ball system with vanishing damping term , 2016 .

[18]  Hedy Attouch,et al.  Fast Proximal Methods via Time Scaling of Damped Inertial Dynamics , 2019, SIAM J. Optim..

[19]  Alexandre Cabot,et al.  Asymptotics for some semilinear hyperbolic equations with non-autonomous damping , 2012 .

[20]  H. Attouch,et al.  Fast convex optimization via time scaling of damped inertial gradient dynamics , 2020 .

[21]  H. Attouch,et al.  Fast convex optimization via inertial dynamics combining viscous and Hessian-driven damping with time rescaling , 2020, Evolution Equations & Control Theory.

[22]  Ya-Ping Fang,et al.  Convergence Rates of Inertial Primal-Dual Dynamical Methods for Separable Convex Optimization Problems , 2020, SIAM J. Control. Optim..

[23]  Othmane Sebbouh,et al.  Convergence Rates of Damped Inertial Dynamics under Geometric Conditions and Perturbations , 2020, SIAM J. Optim..

[24]  Radu Ioan Bot,et al.  A primal-dual dynamical approach to structured convex minimization problems , 2019, 1905.08290.

[25]  Long time behavior for a semilinear hyperbolic equation with asymtotically vanishing damping term and convex potential , 2014, 1412.7008.

[26]  ON A SECOND ORDER DISSIPATIVE ODE IN HILBERT SPACE WITH AN INTEGRABLE SOURCE TERM Dedicated to Professor Constantine M. Dafermos on the occasion of his 70th birthday , 2012 .

[27]  Yangyang Xu,et al.  Accelerated First-Order Primal-Dual Proximal Methods for Linearly Constrained Composite Convex Programming , 2016, SIAM J. Optim..

[28]  Ramzi May.  Asymptotic for a second order evolution equation with convex potential and vanishing damping term , 2015, 1509.05598.

[29]  H. Attouch,et al.  Rate of convergence of inertial gradient dynamics with time-dependent viscous damping coefficient , 2018 .

[30]  Stephen P. Boyd,et al.  A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights , 2014, J. Mach. Learn. Res..

[31]  Jean-François Aujol,et al.  The Differential Inclusion Modeling FISTA Algorithm and Optimality of Convergence Rate in the Case b $\leq3$ , 2018, SIAM J. Optim..

[32]  Alejandro Ribeiro,et al.  A variational approach to dual methods for constrained convex optimization , 2017, 2017 American Control Conference (ACC).

[33]  Michael I. Jordan,et al.  A Lyapunov Analysis of Momentum Methods in Optimization , 2016, ArXiv.

[34]  Yaping Fang,et al.  Convergence rate analysis of fast primal-dual methods with scalings for linearly constrained convex optimization problems ⋆ , 2021 .

[35]  Hedy Attouch.  Fast inertial proximal ADMM algorithms for convex structured optimization with linear constraint , 2020 .

[36]  Zhouchen Lin,et al.  Accelerated Algorithms for Constrained Convex Optimization , 2020, Accelerated Optimization for Machine Learning.

[37]  Euhanna Ghadimi,et al.  Global convergence of the Heavy-ball method for convex optimization , 2014, 2015 European Control Conference (ECC).

[38]  T. Sideris.  Ordinary Differential Equations and Dynamical Systems , 2013 .

[39]  Felipe Alvarez,et al.  On the Minimizing Property of a Second Order Dissipative System in Hilbert Spaces , 2000, SIAM J. Control. Optim..

[40]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[41]  Mohamed Ali Jendoubi,et al.  Asymptotics for a second-order differential equation with nonautonomous damping and an integrable source term , 2015 .

[42]  Boris Polyak.  Some methods of speeding up the convergence of iteration methods , 1964 .

[43]  H. Attouch,et al.  Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3 , 2017, ESAIM: Control, Optimisation and Calculus of Variations.

[44]  Xianlin Zeng,et al.  Dynamical Primal-Dual Accelerated Method with Applications to Network Optimization , 2019 .

[45]  S. Gadat,et al.  On the long time behavior of second order differential equations with asymptotically small dissipation , 2007, 0710.1107.

[46]  Andre Wibisono,et al.  A variational perspective on accelerated methods in optimization , 2016, Proceedings of the National Academy of Sciences.