Fixed-Time Stable Gradient Flows: Applications to Continuous-Time Optimization

Continuous-time optimization is currently an active field of research in optimization theory; prior work in this area has yielded useful insights and elegant methods for proving stability and convergence properties of the continuous-time optimization algorithms. This article proposes novel gradient-flow schemes that yield convergence to the optimal point of a convex optimization problem within a fixed time from any given initial condition for unconstrained optimization, constrained optimization, and min–max problems. It is shown that the solution of the modified gradient-flow dynamics exists and is unique under certain regularity conditions on the objective function, while fixed-time convergence to the optimal point is shown via Lyapunov-based analysis. The application of the modified gradient flow to unconstrained optimization problems is studied under the assumption of gradient dominance, a relaxation of strong convexity. Then, a modified Newton's method is presented that exhibits fixed-time convergence under some mild conditions on the objective function. Building upon this method, a novel technique for solving convex optimization problems with linear equality constraints that yields convergence to the optimal point in fixed time is developed. Finally, the general min–max problem is considered, and a modified saddle-point dynamics to obtain the optimal solution in fixed time is developed.

[1]  丸山 徹 Convex Analysisの二,三の進展について , 1977 .

[2]  David A. Wismer,et al.  Introduction to nonlinear optimization , 1978 .

[3]  M. Bartholomew-Biggs,et al.  Some effective methods for unconstrained optimization based on the solution of systems of ordinary differential equations , 1989 .

[4]  V. Lakshmikantham,et al.  Uniqueness and nonuniqueness criteria for ordinary differential equations , 1993 .

[5]  U. Helmke,et al.  Optimization and Dynamical Systems , 1994, Proceedings of the IEEE.

[6]  Dennis S. Bernstein,et al.  Finite-Time Stability of Continuous Autonomous Systems , 2000, SIAM J. Control. Optim..

[7]  J. Schropp,et al.  A dynamical systems approach to constrained minimization , 2000 .

[8]  Martin Rumpf,et al.  Relations between Optimization and Gradient Flow Methods with Applications to Image Registration , 2001 .

[9]  C. Zălinescu Convex analysis in general vector spaces , 2002 .

[10]  Gene H. Golub,et al.  Numerical solution of saddle point problems , 2005, Acta Numerica.

[11]  Laura Palagi,et al.  Quartic Formulation of Standard Quadratic Optimization Problems , 2005, J. Glob. Optim..

[12]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[13]  Jorge Cortés,et al.  Finite-time convergent gradient flows with applications to network consensus , 2006, Autom..

[14]  Christopher C. Paige,et al.  HUA'S MATRIX EQUALITY AND SCHUR COMPLEMENTS , 2008 .

[15]  Yurii Nesterov,et al.  Accelerating the cubic regularization of Newton’s method on convex problems , 2005, Math. Program..

[16]  Fernando Paganini,et al.  Stability of primal-dual gradient dynamics and applications to network optimization , 2010, Autom..

[17]  Jing Wang,et al.  A control perspective for centralized and distributed convex optimization , 2011, IEEE Conference on Decision and Control and European Control Conference.

[18]  W. Marsden I and J , 2012 .

[19]  Andrey Polyakov,et al.  Nonlinear Feedback Design for Fixed-Time Stabilization of Linear Control Systems , 2012, IEEE Transactions on Automatic Control.

[20]  C. Ebenbauer,et al.  On a Class of Smooth Optimization Algorithms with Applications in Control , 2012 .

[21]  Bahman Gharesifard,et al.  Distributed convergence to Nash equilibria in two-network zero-sum games , 2012, Autom..

[22]  Nicola Elia,et al.  A distributed continuous-time gradient dynamics approach for the active power loss minimizations , 2013, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[23]  S. Osher,et al.  Sparse Recovery via Differential Inclusions , 2014, 1406.7728.

[24]  Alexander G. Loukianov,et al.  A fixed time convergent dynamical system to solve linear programming , 2014, 53rd IEEE Conference on Decision and Control.

[25]  Amir Beck,et al.  Introduction to Nonlinear Optimization - Theory, Algorithms, and Applications with MATLAB , 2014, MOS-SIAM Series on Optimization.

[26]  M. Durea,et al.  An Introduction to Nonlinear Optimization Theory , 2014 .

[27]  Alexandre M. Bayen,et al.  Accelerated Mirror Descent in Continuous and Discrete Time , 2015, NIPS.

[28]  Ashia C. Wilson,et al.  On Accelerated Methods in Optimization , 2015, 1509.03616.

[29]  Weisheng Chen,et al.  Finite-time convergent distributed consensus optimisation over networks , 2016 .

[30]  Mark W. Schmidt,et al.  Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition , 2016, ECML/PKDD.

[31]  Martin J. Wainwright,et al.  Minimax Optimal Procedures for Locally Private Estimation , 2016, ArXiv.

[32]  Stephen P. Boyd,et al.  A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights , 2014, J. Mach. Learn. Res..

[33]  Andre Wibisono,et al.  A variational perspective on accelerated methods in optimization , 2016, Proceedings of the National Academy of Sciences.

[34]  Xiaojun Zhou,et al.  A fixed time distributed optimization: A sliding mode perspective , 2017, IECON 2017 - 43rd Annual Conference of the IEEE Industrial Electronics Society.

[35]  ASHISH CHERUKURI,et al.  Saddle-Point Dynamics: Conditions for Asymptotic Stability of Saddle Points , 2015, SIAM J. Control. Optim..

[36]  Guoqiang Hu,et al.  Finite-time distributed optimization with quadratic objective functions under uncertain information , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[37]  R. Sarpong,et al.  Bio-inspired synthesis of xishacorenes A, B, and C, and a new congener from fuscol† †Electronic supplementary information (ESI) available. See DOI: 10.1039/c9sc02572c , 2019, Chemical science.

[38]  Andrey Polyakov,et al.  Consistent Discretization of Finite-time Stable Homogeneous Systems , 2018, 2018 15th International Workshop on Variable Structure Systems (VSS).

[39]  Zengqiang Chen,et al.  Distributed Optimization with Finite-Time Convergence via Discontinuous Dynamics , 2018, 2018 37th Chinese Control Conference (CCC).

[40]  Enrique Mallada,et al.  The Role of Convexity in Saddle-Point Dynamics: Lyapunov Function and Robustness , 2016, IEEE Transactions on Automatic Control.

[41]  Wei Ren,et al.  Convex Optimization via Finite-Time Projected Gradient Flows , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[42]  Andrea Gasparri,et al.  A Finite-Time Protocol for Distributed Continuous-Time Optimization of Sum of Locally Coupled Strictly Convex Functions , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[43]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[44]  R. Vasudevan,et al.  Fixed-Time Stable Proximal Dynamical System for Solving Mixed Variational Inequality Problems , 2019 .

[45]  Mihailo R. Jovanovic,et al.  The Proximal Augmented Lagrangian Method for Nonsmooth Composite Optimization , 2016, IEEE Transactions on Automatic Control.

[46]  Andrey Polyakov,et al.  Consistent Discretization of Finite-Time and Fixed-Time Stable Systems , 2019, SIAM J. Control. Optim..

[47]  Na Li,et al.  On the Exponential Stability of Primal-Dual Gradient Dynamics , 2018, IEEE Control Systems Letters.