From Contraction Theory to Fixed Point Algorithms on Riemannian and Non-Euclidean Spaces

The design of fixed point algorithms is at the heart of monotone operator theory, convex analysis, and of many modern optimization problems arising in machine learning and control. This tutorial reviews recent advances in understanding the relationship between Demidovich conditions, one-sided Lipschitz conditions, and contractivity theorems. We review the standard contraction theory on Euclidean spaces as well as little-known results for Riemannian manifolds. Special emphasis is placed on the setting of non-Euclidean norms and the recently introduced weak pairings for the `1 and `∞ norms. We highlight recent results on explicit and implicit fixed point schemes for non-Euclidean contracting systems.

[1]  Zahra Aminzare,et al.  Logarithmic Lipschitz norms and diffusion-induced instability , 2012, Nonlinear analysis, theory, methods & applications.

[2]  O. P. Ferreira,et al.  Contributions to the Study of Monotone Vector Fields , 2002 .

[3]  Winfried Stefan Lohmiller,et al.  Contraction analysis of nonlinear systems , 1999 .

[4]  Francesco Bullo,et al.  Non-Euclidean Contraction Theory for Robust Nonlinear Stability , 2021 .

[5]  Zahra Aminzarey,et al.  Contraction methods for nonlinear systems: A brief introduction and some open problems , 2014, 53rd IEEE Conference on Decision and Control.

[6]  Sandor Nemeth.  Geodesic monotone vector fields. , 1999 .

[7]  Francesco Bullo,et al.  Robust Implicit Networks via Non-Euclidean Contractions , 2021, NeurIPS.

[8]  J. H. Wang,et al.  Monotone and Accretive Vector Fields on Riemannian Manifolds , 2010 .

[9]  Francesco Bullo,et al.  Non-Euclidean Contraction Theory for Monotone and Positive Systems , 2021 .

[10]  Jean-Jacques Slotine,et al.  Beyond convexity—Contraction and global convergence of gradient descent , 2020, PloS one.

[11]  Francesco Bullo,et al.  A Contraction Theory Approach to Optimization Algorithms from Acceleration Flows , 2021, ArXiv.

[12]  Francesco Bullo,et al.  Contraction theory on Riemannian manifolds , 2014, Syst. Control. Lett..

[13]  N. K. Sahu,et al.  Approximation solvability of a class of A-monotone implicit variational inclusion problems in semi-inner product spaces , 2014, Appl. Math. Comput..

[14]  Stephen P. Boyd,et al.  A Primer on Monotone Operator Methods , 2015 .

[15]  C. Desoer,et al.  The measure of a matrix as a tool to analyze computer algorithms for circuit analysis , 1972 .

[16]  E. Hairer,et al.  Solving Ordinary Differential Equations II: Stiff and Differential-Algebraic Problems , 2010 .

[17]  Patrick L. Combettes,et al.  Fixed Point Strategies in Data Science , 2020 .

[18]  Ian R. Manchester,et al.  Lipschitz Bounded Equilibrium Networks , 2020, ArXiv.