Generating trading agent strategies: Analytic and empirical methods for infinite and large games

A Strategy Generation Engine is a system that reads a description of a game or market mechanism and outputs strategies for participants. Ideally, this means a game solver---an algorithm to compute Nash equilibria. This is a well-studied problem and very general solutions exist, but they can only be applied to small, finite games. This thesis presents methods for finding or approximating Nash equilibria for infinite games, and for intractably large finite games. First, I define a broad class of one-shot, two-player infinite games of incomplete information. I present an algorithm for computing best-response strategies in this class and show that for many particular games the algorithm can be iterated to compute Nash equilibria. Many results from the game theory literature are reproduced---automatically---using this method, as well as novel results for new games. Next, I address the problem of finding strategies in games that, even if finite, are larger than what any exact solution method can address. Our solution involves (1) generating a small set of candidate strategies, (2) constructing via simulation an approximate payoff matrix for the simplification of the game restricted to the candidate strategies, (3) analyzing the empirical game, and (4) assessing the quality of solutions with respect to the underlying full game. I leave methods for generating candidate strategies domain-specific and focus on methods for taming the computational cost of empirical game generation. I do this by employing Monte Carlo variance reduction techniques and introducing a technique for approximating many-player games by reducing the number of players. We are additionally able to solve much larger payoff matrices than the current state-of-the-art solver by exploiting symmetry in games. I test these methods in small games with known solutions and then apply them to two realistic market scenarios: Simultaneous Ascending Auctions (SAA) and the Trading Agent Competition (TAC) travel-shopping game. For these domains I focus on two key price prediction approaches for generating candidate strategies for empirical analysis: self-confirming price predictions and Walrasian equilibrium prices. We find that these are highly effective strategies in SAA and TAC, respectively.

[1]  A. Cournot Researches into the Mathematical Principles of the Theory of Wealth , 1898, Forerunners of Realizable Values Accounting in Financial Reporting.

[2]  J. Neumann,et al.  Theory of Games and Economic Behavior. , 1945 .

[3]  E. Rowland Theory of Games and Economic Behavior , 1946, Nature.

[4]  Claude E. Shannon,et al.  Programming a computer for playing chess , 1950 .

[5]  J. Robinson AN ITERATIVE METHOD OF SOLVING A GAME , 1951, Classics in Game Theory.

[6]  O. H. Brownlee,et al.  ACTIVITY ANALYSIS OF PRODUCTION AND ALLOCATION , 1952 .

[7]  Gerard Debreu,et al.  A Social Equilibrium Existence Theorem* , 1952, Proceedings of the National Academy of Sciences.

[8]  William Vickrey,et al.  Counterspeculation, Auctions, And Competitive Sealed Tenders , 1961 .

[9]  C. E. Lemke,et al.  Equilibrium Points of Bimatrix Games , 1964 .

[10]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[11]  S. Vajda Some topics in two-person games , 1971 .

[12]  K. Arrow,et al.  General Competitive Analysis , 1971 .

[13]  M. Bacharach Two-person Cooperative Games , 1976 .

[14]  R. Lewontin ‘The Selfish Gene’ , 1977, Nature.

[15]  P. Taylor,et al.  Evolutionarily Stable Strategies and Game Dynamics , 1978 .

[16]  Paul R. Milgrom,et al.  A theory of auctions and competitive bidding , 1982 .

[17]  V. Crawford,et al.  Job Matching, Coalition Formation, and Gross Substitutes , 1982 .

[18]  E. Damme Refinements of the Nash Equilibrium Concept , 1983 .

[19]  Kalyan Chatterjee,et al.  Bargaining under Incomplete Information , 1983, Oper. Res..

[20]  Robert J. Weber,et al.  Distributional Strategies for Games with Incomplete Information , 1985, Math. Oper. Res..

[21]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.

[22]  R. McAfee,et al.  Auctions and Bidding , 1986 .

[23]  Roy Radner,et al.  Equilibria of the Sealed- Bid Mechanism for Bargaining with Incomplete Information*, ' , 1987 .

[24]  J. Rice Mathematical Statistics and Data Analysis , 1988 .

[25]  W. Arthur,et al.  The Economy as an Evolving Complex System II , 1988 .

[26]  John Rust,et al.  The Double Auction Market , 1989 .

[27]  Steven R. Williams,et al.  Bilateral trade with the sealed bid k-double auction: Existence and efficiency , 1989 .

[28]  J. Boyd Symmetries, Dynamic Equilibria, and the Value Function , 1990 .

[29]  David M. Kreps,et al.  Game Theory and Economic Modelling , 1992 .

[30]  Paul R. Milgrom,et al.  Adaptive and sophisticated learning in normal form games , 1991 .

[31]  D. Friedman EVOLUTIONARY GAMES IN ECONOMICS , 1991 .

[32]  Mark A. Satterthwaite,et al.  The Bayesian theory of the k-double auction: Santa Fe Institute Studies in the Sciences of Complexity , 2018 .

[33]  Dimitri P. Bertsekas,et al.  Auction algorithms for network flow problems: A tutorial introduction , 1992, Comput. Optim. Appl..

[34]  J. Kagel,et al.  Independent Private Value Auctions: Bidder Behaviour in First-, Second- and Third-Price Auctions with Varying Numbers of Bidders , 1993 .

[35]  Heinz Kredel,et al.  Gröbner Bases: A Computational Approach to Commutative Algebra , 1993 .

[36]  Pierre L'Ecuyer,et al.  Efficiency improvement and variance reduction , 1994, Proceedings of Winter Simulation Conference.

[37]  R. Rubinstein,et al.  On relaxation algorithms in computation of noncooperative equilibria , 1994, IEEE Trans. Autom. Control..

[38]  Jeffrey S. Rosenschein,et al.  Rules of Encounter - Designing Conventions for Automated Negotiation among Computers , 1994 .

[39]  Jörgen W. Weibull,et al.  Evolutionary Game Theory , 1996 .

[40]  J. Morgan,et al.  An Analysis of the War of Attrition and the All-Pay Auction , 1997 .

[41]  R. Selten,et al.  Experimental Sealed Bid First Price Auctions with Directly Observed Bid Functions , 1994 .

[42]  P. Cramton Money Out of Thin Air: The Nationwide Narrowband PCS Auction , 1995 .

[43]  N. Bose Gröbner Bases: An Algorithmic Method in Polynomial Ideal Theory , 1995 .

[44]  Ronald Fagin,et al.  Reasoning about knowledge , 1995 .

[45]  A. Mas-Colell,et al.  Microeconomic Theory , 1995 .

[46]  R. Palmer,et al.  Asset Pricing Under Endogenous Expectations in an Artificial Stock Market , 1996 .

[47]  D. Koller,et al.  Efficient Computation of Equilibria for Extensive Two-Person Games , 1996 .

[48]  R. McKelvey,et al.  Computation of equilibria in finite games , 1996 .

[49]  R. McAfee,et al.  Analyzing the Airwaves Auction , 1996 .

[50]  Dov Monderer,et al.  A Learning Approach to Auctions , 1998 .

[51]  H. Kuk On equilibrium points in bimatrix games , 1996 .

[52]  R. Weber Making More from Less: Strategic Demand Reduction in the FCC Spectrum Auctions , 1997 .

[53]  Tony Curzon Price,et al.  Using co-evolutionary programming to simulate strategic behaviour in markets , 1997 .

[54]  Avi Pfeffer,et al.  Representations and Solutions for Game-Theoretic Problems , 1997, Artif. Intell..

[55]  S. Bikhchandani,et al.  Competitive Equilibrium in an Exchange Economy with Indivisibilities , 1997 .

[56]  Susan Athey,et al.  Single Crossing Properties and the Existence of Pure Strategy Equilibria in Games of Incomplete Information , 1997 .

[57]  Jeffrey O. Kephart,et al.  Price and Niche Wars in a Free-Market Economy of Software Agents , 1997, Artificial Life.

[58]  Jean-Francois Richard,et al.  Empirical Game Theoretic Models: Constrained Equilibrium & Simulation , 1998 .

[59]  Michael B. Gordy Computationally Convenient Distributional Assumptions for Common-Value Auctions , 1997 .

[60]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[61]  Eric van Damme,et al.  Non-Cooperative Games , 2000 .

[62]  Yoav Shoham,et al.  Towards a universal test suite for combinatorial auction algorithms , 2000, EC '00.

[63]  Blake LeBaron,et al.  Agent-based computational finance : Suggested readings and early research , 2000 .

[64]  Pierfrancesco La Mura Game Networks , 2000, UAI.

[65]  Paul R. Milgrom,et al.  Putting Auction Theory to Work: The Simultaneous Ascending Auction , 1999, Journal of Political Economy.

[66]  Michael P. Wellman,et al.  Combinatorial auctions for supply chain formation , 2000, EC '00.

[67]  Peter Stone,et al.  ATTac-2000: an adaptive autonomous bidding agent , 2001, AGENTS '01.

[68]  Michael L. Littman,et al.  Graphical Models for Game Theory , 2001, UAI.

[69]  Shou-De Lin,et al.  Designing the Market Game for a Trading Agent Competition , 2001, IEEE Internet Comput..

[70]  Michael P. Wellman,et al.  Auction Protocols for Decentralized Scheduling , 2001, Games Econ. Behav..

[71]  Michael P. Wellman,et al.  Market protocols for decentralized supply chain formation , 2001 .

[72]  Peter Stone,et al.  FAucS : An FCC Spectrum Auction Simulator for Autonomous Bidding Agents , 2001, WELCOM.

[73]  Leigh Tesfatsion,et al.  Market power and efficiency in a computational electricity market with discriminatory double-auction pricing , 2001, IEEE Trans. Evol. Comput..

[74]  Gerhard Weiß,et al.  Antisocial Agents and Vickrey Auctions , 2001, ATAL.

[75]  Flavio M. Menezes,et al.  Private provision of discrete public goods with incomplete information , 2001 .

[76]  Amy Greenwald,et al.  Bid determination in simultaneous actions an agent architecture , 2001, EC '01.

[77]  Peter Stone,et al.  ATTac-2001: A Learning, Autonomous Bidding Agent , 2002, AMEC.

[78]  Nicholas R. Jennings,et al.  SouthamptonTAC: Designing a Successful Trading Agent , 2002, ECAI.

[79]  Elizabeth Sklar,et al.  Co-Evolution of Auction Mechanisms and Trading Strategies: Towards a Novel Approach to Microeconomic , 2002 .

[80]  G. Tesauro,et al.  Analyzing Complex Strategic Interactions in Multi-Agent Systems , 2002 .

[81]  Peter Stone Multiagent Competitions and Research: Lessons from RoboCup and TAC , 2002, RoboCup.

[82]  Robert Wilson,et al.  Structure theorems for game trees , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[83]  William H. Press,et al.  Numerical recipes in C , 2002 .

[84]  Michael P. Wellman,et al.  The 2001 trading agent competition , 2002, Electron. Mark..

[85]  Klaus Dorer,et al.  Agent-oriented software engineering for successful TAC participation , 2002, AAMAS '02.

[86]  David A. McAllester,et al.  Decision-Theoretic Bidding Based on Learned Density Models in Simultaneous, Interacting Auctions , 2003, J. Artif. Intell. Res..

[87]  J. Morgan,et al.  The Spite Motive and Equilibrium Behavior in Auctions , 2003 .

[88]  Daphne Koller,et al.  A Continuation Method for Nash Equilibria in Structured Games , 2003, IJCAI.

[89]  Brian Gough,et al.  GNU Scientific Library Reference Manual - Third Edition , 2003 .

[90]  Andrew Byde,et al.  Applying evolutionary game theory to auction mechanism design , 2003, EEE International Conference on E-Commerce, 2003. CEC 2003..

[91]  Rajarshi Das,et al.  Choosing Samples to Compute Heuristic-Strategy Nash Equilibrium , 2003, AMEC.

[92]  Robert Wilson,et al.  A global Newton method to compute Nash equilibria , 2003, J. Econ. Theory.

[93]  P. Bose,et al.  Fast approximations for sums of distances, clustering and the Fermat-Weber problem , 2003, Comput. Geom..

[94]  Dave Cliff,et al.  Explorations in evolutionary design of online auction market mechanisms , 2003, Electron. Commer. Res. Appl..

[95]  Michael P. Wellman,et al.  Trading Agents Competing: Performance, Progress, and Market Effectiveness , 2003, IEEE Intell. Syst..

[96]  Bart Selman,et al.  A principled study of the design tradeoffs for autonomous trading agents , 2003, AAMAS '03.

[97]  Moshe Tennenholtz,et al.  Local-Effect Games , 2003, IJCAI.

[98]  Michael P. Wellman,et al.  Exploring bidding strategies for market-based scheduling , 2003, EC '03.

[99]  Aranyak Mehta,et al.  Playing large games using simple strategies , 2003, EC '03.

[100]  Sven de Vries,et al.  Combinatorial Auctions: A Survey , 2003, INFORMS J. Comput..

[101]  Nicholas R. Jennings,et al.  SouthamptonTAC: An adaptive autonomous trading agent , 2003, TOIT.

[102]  M. P. Wellman,et al.  Price Prediction in a Trading Agent Competition , 2004, J. Artif. Intell. Res..

[103]  Michael P. Wellman,et al.  Price Prediction Strategies for Market-Based Scheduling , 2004, ICAPS.

[104]  Jeffrey O. Kephart,et al.  Shopbot Economics , 2004, Autonomous Agents and Multi-Agent Systems.

[105]  P. Cramton Simultaneous Ascending Auctions , 2004 .

[106]  Kevin Leyton-Brown,et al.  Computing Nash Equilibria of Action-Graph Games , 2004, UAI.

[107]  Michael P. Wellman,et al.  Computing approximate bayes-nash equilibria in tree-games of incomplete information , 2004, EC '04.

[108]  Michael P. Wellman,et al.  Computing Best-Response Strategies in Infinite Games of Incomplete Information , 2004, UAI.

[109]  Daniel M. Reeves,et al.  Notes on Equilibria in Symmetric Games , 2004 .

[110]  Yoav Shoham,et al.  Run the GAMUT: a comprehensive approach to evaluating game-theoretic algorithms , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[111]  Amy Greenwald,et al.  Bidding under Uncertainty: Theory and Experiments , 2004, UAI.

[112]  Michael P. Wellman,et al.  Searching for Walverine 2005 , 2005, AMEC@AAMAS/TADA@IJCAI.

[113]  Peter Stone,et al.  The First International Trading Agent Competition: Autonomous Bidding Agents , 2005, Electron. Commer. Res..

[114]  Michael P. Wellman,et al.  STRATEGIC INTERACTIONS IN A SUPPLY CHAIN GAME , 2005, Comput. Intell..

[115]  Michael P. Wellman,et al.  Self-Confirming Price Prediction for Bidding in Simultaneous Ascending Auctions , 2005, UAI.

[116]  Michael P. Wellman,et al.  Walverine: a Walrasian trading agent , 2003, AAMAS '03.

[117]  Michael P. Wellman,et al.  Approximate Strategic Reasoning through Hierarchical Reduction of Large Symmetric Games , 2005, AAAI.

[118]  M. Wellman,et al.  Automated Markets and Trading Agents , 2007 .

[119]  E. Maasland,et al.  Auction Theory , 2021, Springer Texts in Business and Economics.

[120]  Michael Peters,et al.  Internet Auctions with Many Traders , 2002, J. Econ. Theory.

[121]  Yoav Shoham,et al.  Spiteful Bidding in Sealed-Bid Auctions , 2007, IJCAI.

[122]  Michel Gendreau,et al.  Combinatorial auctions , 2007, Ann. Oper. Res..

[123]  이성수,et al.  Simulation , 2006, Healthcare Simulation at a Glance.