Learning equilibria in symmetric auction games using artificial neural networks

Auction theory is of central importance in the study of markets. Unfortunately, we do not know equilibrium bidding strategies for most auction games. For realistic markets with multiple items and value interdependencies the Bayes-Nash equilibria often turn out to be intractable systems of partial differential equations. We use neural networks and self-play to provably learn local equilibria in such auction games. Our empirical results show that these approximated Bayes-Nash equilibria coincide with the global equilibria whenever available. The method follows the simultaneous gradient of the game and uses a smoothing technique to circumvent discontinuities in the ex-post utility functions of auction games. This can be explained by the fact that bidders in most auction models are symmetric which leads to potential games for which gradient dynamics converge.

[1]  Yang Cai,et al.  Simultaneous bayesian auctions and computational complexity , 2014, EC.

[2]  David Silver,et al.  A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning , 2017, NIPS.

[3]  Sven Seuken,et al.  Deep Learning-powered Iterative Combinatorial Auctions , 2019, AAAI.

[4]  Richard P. McLean,et al.  Optimal Selling Strategies under Uncertainty for a Discriminating Monopolist When Demands Are Interdependent , 1985 .

[5]  R. Ghani.  Predicting the End-Price of Online Auctions , 2004 .

[6]  Éva Tardos,et al.  Learning in Games: Robustness of Fast Convergence , 2016, NIPS.

[7]  Demis Hassabis,et al.  A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play , 2018, Science.

[8]  Gunes Ercal,et al.  On No-Regret Learning, Fictitious Play, and Nash Equilibrium , 2001, ICML.

[9]  P. Klemperer.  Auction Theory: A Guide to the Literature , 1999 .

[10]  E. Maasland,et al.  Auction Theory , 2021, Springer Texts in Business and Economics.

[11]  Wolfgang Hörmann,et al.  Automatic Nonuniform Random Variate Generation , 2011 .

[12]  J. Goeree,et al.  Handbook of Spectrum Auction Design , 2017 .

[13]  Andriy Zapechelnyuk,et al.  No-regret dynamics and fictitious play , 2012, J. Econ. Theory.

[14]  Gábor Lugosi,et al.  Learning correlated equilibria in games with compact sets of strategies , 2007, Games Econ. Behav..

[15]  David C. Parkes,et al.  Deep Learning for Revenue-Optimal Auctions with Budgets , 2018, AAMAS.

[16]  Paul W. Goldberg,et al.  The complexity of computing a Nash equilibrium , 2006, STOC '06.

[17]  William Vickrey,et al.  Counterspeculation, Auctions, And Competitive Sealed Tenders , 1961 .

[18]  Takashi Ui,et al.  Bayesian Nash equilibrium and variational inequalities , 2016 .

[19]  Paul Dütting,et al.  Optimal auctions through deep learning , 2017, ICML.

[20]  Jacob K. Goeree,et al.  On the impossibility of core-selecting auctions: Impossibility of core-selecting auctions , 2016 .

[21]  Victor Naroditskiy,et al.  Using Iterated Best-Response to Find Bayes-Nash Equilibria in Auctions , 2007, AAAI.

[22]  J. I. Ramos,et al.  Introduction to Hilbert spaces with applications , 1992 .

[23]  Xi Chen,et al.  Evolution Strategies as a Scalable Alternative to Reinforcement Learning , 2017, ArXiv.

[24]  L. Shapley,et al.  REGULAR ARTICLEPotential Games , 1996 .

[25]  P. Reny,et al.  AN EFFICIENT AUCTION , 2002 .

[26]  D. Fudenberg,et al.  Learning and Equilibrium , 2009 .

[27]  Robert B. Wilson.  Competitive Bidding with Disparate Information , 2007 .

[28]  Constantinos Daskalakis,et al.  The complexity of constrained min-max optimization , 2020, STOC.

[29]  Maarten C. W. Janssen.  Reflections on the 2020 Nobel Memorial Prize Awarded to Paul Milgrom and Robert Wilson , 2021 .

[30]  Josef Hofbauer,et al.  Perturbations of Set-Valued Dynamical Systems, with Applications to Game Theory , 2012, Dyn. Games Appl..

[31]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.

[32]  Sven Seuken,et al.  Computing Bayes-Nash Equilibria in Combinatorial Auctions with Continuous Value and Action Spaces , 2017, IJCAI.

[33]  Michael P. Wellman,et al.  Computing Equilibrium Strategies in Infinite Games of Incomplete Information , 2022 .

[34]  D. Bergemann,et al.  Robust Implementation in Direct Mechanisms , 2009 .

[35]  Kevin Leyton-Brown,et al.  Deep Learning for Predicting Human Strategic Behavior , 2016, NIPS.

[36]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[37]  Tuomas Sandholm,et al.  Deep Counterfactual Regret Minimization , 2018, ICML.

[38]  Amy Greenwald,et al.  Empirical Mechanism Design: Designing Mechanisms from Data , 2019, UAI.

[39]  Michael P. Wellman,et al.  Constrained automated mechanism design for infinite games of incomplete information , 2007, Autonomous Agents and Multi-Agent Systems.

[40]  Quang Vuong,et al.  Asymmetry in first-price auctions with affiliated private values , 2003 .

[41]  J. Goeree,et al.  On the Impossibility of Core-Selecting Auctions , 2009 .

[42]  Nicholas R. Jennings,et al.  Computing pure Bayesian-Nash equilibria in games with finite actions and continuous types , 2013, Artif. Intell..

[43]  Thore Graepel,et al.  A Neural Architecture for Designing Truthful and Efficient Auctions , 2019, ArXiv.

[44]  Michael Bowling,et al.  Hindsight and Sequential Rationality of Correlated Play , 2021, AAAI.

[45]  Paul W. Goldberg,et al.  The Complexity of Computing a Nash Equilibrium , 2009, SIAM J. Comput..

[46]  Martin Bichler,et al.  Split‐Award Procurement Auctions—Can Bayesian Equilibrium Strategies Predict Human Bidding Behavior in Multi‐Object Auctions? , 2015 .

[47]  Michael H. Bowling,et al.  Convergence and No-Regret in Multiagent Learning , 2004, NIPS.

[48]  Sven Seuken,et al.  Designing Core-selecting Payment Rules: A Computational Search Approach , 2018, EC.

[49]  Thore Graepel,et al.  Differentiable Game Mechanics , 2019, J. Mach. Learn. Res..

[50]  Paul R. Milgrom,et al.  A theory of auctions and competitive bidding , 1982 .

[51]  S. Shankar Sastry,et al.  On Gradient-Based Learning in Continuous Games , 2018, SIAM J. Math. Data Sci..

[52]  Éva Tardos,et al.  No-Regret Learning in Bayesian Games , 2015, NIPS.

[53]  Noam Brown,et al.  Superhuman AI for multiplayer poker , 2019, Science.

[54]  Lawrence M. Ausubel,et al.  Demand Reduction and Inefficiency in Multi-Unit Auctions , 2014 .

[55]  B. Moldovanu,et al.  The Limits of ex post Implementation , 2006 .

[56]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[57]  David Silver,et al.  Deep Reinforcement Learning from Self-Play in Imperfect-Information Games , 2016, ArXiv.

[58]  Sven Seuken,et al.  Computing Bayes-Nash Equilibria in Combinatorial Auctions with Verification , 2018, J. Artif. Intell. Res..

[59]  Oleg V. Baranov,et al.  Core-selecting auctions with incomplete information , 2020, Int. J. Game Theory.

[60]  Martin Bichler,et al.  Ascending Combinatorial Auctions with Risk Averse Bidders , 2016 .