Competitive Mirror Descent

Constrained competitive optimization involves multiple agents trying to minimize conflicting objectives, subject to constraints. This is a highly expressive modeling language that subsumes most of modern machine learning. In this work we propose competitive mirror descent (CMD): a general method for solving such problems based on first order information that can be obtained by automatic differentiation. First, by adding Lagrange multipliers, we obtain a simplified constraint set with an associated Bregman potential. At each iteration, we then solve for the Nash equilibrium of a regularized bilinear approximation of the full problem to obtain a direction of movement of the agents. Finally, we obtain the next iterate by following this direction according to the dual geometry induced by the Bregman potential. By using the dual geometry we obtain feasible iterates despite only solving a linear system at each iteration, eliminating the need for projection steps while still accounting for the global nonlinear structure of the constraint set. As a special case we obtain a novel competitive multiplicative weights algorithm for problems on the positive cone.

[1]  Sayan Mukherjee,et al.  The Information Geometry of Mirror Descent , 2013, IEEE Transactions on Information Theory.

[2]  Sridhar Mahadevan,et al.  Global Convergence to the Equilibrium of GANs using Variational Inequalities , 2018, ArXiv.

[3]  Max Welling,et al.  Group Equivariant Convolutional Networks , 2016, ICML.

[4]  Stefano Ermon,et al.  Label-Free Supervision of Neural Networks with Physics and Domain Knowledge , 2016, AAAI.

[5]  Florian Schäfer,et al.  Competitive Gradient Descent , 2019, NeurIPS.

[6]  Karthik Sridharan,et al.  Two-Player Games for Efficient Non-Convex Constrained Optimization , 2018, ALT.

[7]  Maya R. Gupta,et al.  Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints , 2018, ICML.

[8]  Aleksander Madry,et al.  Towards Deep Learning Models Resistant to Adversarial Attacks , 2017, ICLR.

[9]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[10]  Harikrishna Narasimhan,et al.  Optimizing Generalized Rate Metrics with Three Players , 2019, NeurIPS.

[11]  Sebastian Nowozin,et al.  The Numerics of GANs , 2017, NIPS.

[12]  Thore Graepel,et al.  The Mechanics of n-Player Differentiable Games , 2018, ICML.

[13]  Thore Graepel,et al.  Differentiable Game Mechanics , 2019, J. Mach. Learn. Res..

[14]  Barak A. Pearlmutter.  Fast Exact Multiplication by the Hessian , 1994, Neural Computation.

[15]  Pieter Abbeel,et al.  Constrained Policy Optimization , 2017, ICML.

[16]  Risi Kondor,et al.  Cormorant: Covariant Molecular Neural Networks , 2019, NeurIPS.

[17]  Miroslav Dudík,et al.  Reinforcement Learning with Convex Constraints , 2019, NeurIPS.

[18]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[19]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[20]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[21]  Anima Anandkumar,et al.  Implicit competitive regularization in GANs , 2020, ICML.

[22]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[23]  A. Iusem.  On the convergence properties of the projected gradient method for convex optimization , 2003 .

[24]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[25]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[26]  F. Opitz.  Information geometry and its applications , 2012, 2012 9th European Radar Conference.

[27]  G. M. Korpelevich.  The extragradient method for finding saddle points and other problems , 1976 .

[28]  Pierre-Luc Bacon,et al.  A Lagrangian Method for Inverse Problems in Reinforcement Learning , 2019 .