Mechanism Design and Analysis Using Simulation-Based Game Models

As agent technology matures, it becomes easier to envision electronic marketplaces teeming with autonomous agents. Since agents are explicitly programmed to optimally compete in these marketplaces (within bounds of computational tractability), and markets themselves are designed with specific objectives in mind, tools are necessary for systematic analyses of strategic interactions among autonomous agents. While traditional game-theoretic approaches to the analysis of multi-agent systems can provide much insight, they are often inadequate, as they rely heavily on analytic tractability of the problem at hand; however, even mildly realistic models of electronic marketplaces contain enough complexity to render a fully analytic approach hopeless. To address questions not amenable to traditional theoretical approaches, I develop methods that allow systematic computational analysis of game-theoretic models in which the players' payoff functions are represented using simulations (i.e., simulation-based games). I develop a globally convergent algorithm for Nash equilibrium approximation in infinite simulation-based games, which I instantiate in the context of infinite games of incomplete information. Additionally, I use statistical learning techniques to improve the quality of Nash equilibrium approximation based on data collected from a game simulator. I also derive probabilistic confidence bounds and present convergence results about solutions of finite games modeled using simulations. The former allow an analyst to make statistically-founded statements about results based on game-theoretic simulations, while the latter provide formal justification for approximating game-theoretic solutions using simulation experiments. To address the broader mechanism design problem, I introduce an iterative algorithm for search in the design space, which requires a game solver as a subroutine. As a result, I enable computational mechanism design using simulation-based models of games by availing the designer of a set of solution tools geared specifically towards games modeled using simulations. I apply the developed computational techniques to analyze strategic procurement and answer design questions in a supply-chain simulation, as well as to analyze dynamic bidding strategies in sponsored search auctions. Indeed, the techniques I develop have broad potential applicability beyond electronic marketplaces; they are geared towards any system that features competing strategic players who respond to incentives in a way that can be reasonably predicted via a game-theoretic analysis.

[1]  A. Roth,et al.  Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria , 1998 .

[2]  E. Balder Remarks on Nash equilibria for games with additively coupled payoffs , 1995 .

[3]  E. Rowland Theory of Games and Economic Behavior , 1946, Nature.

[4]  Pierre L'Ecuyer,et al.  Efficiency improvement and variance reduction , 1994, Proceedings of Winter Simulation Conference.

[5]  R. Rosenthal Rules of thumb in games , 1993 .

[6]  T. Sandholm,et al.  Applications of Automated Mechanism Design , 2003 .

[7]  C. d'Aspremont,et al.  Incentives and incomplete information , 1979 .

[8]  Jean-Francois Richard,et al.  Approximation of Bayesian Nash Equilibrium , 2008 .

[9]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[10]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[11]  Michael P. Wellman,et al.  Mechanism Design Based on Beliefs about Responsive Play ( Position Paper ) , 2006 .

[12]  Michael P. Wellman,et al.  Searching for Walverine 2005 , 2005, AMEC@AAMAS/TADA@IJCAI.

[13]  Luis E. Ortiz,et al.  Nash Propagation for Loopy Graphical Games , 2002, NIPS.

[14]  R. Selten Evolution, learning, and economic behavior , 1991 .

[15]  Michael P. Wellman,et al.  Knowledge Combination in Graphical Multiagent Models , 2008, UAI.

[16]  J. Morgan,et al.  The Spite Motive and Equilibrium Behavior in Auctions , 2003 .

[17]  Michael L. Littman,et al.  Graphical Models for Game Theory , 2001, UAI.

[18]  David C. Parkes,et al.  Iterative Combinatorial Auctions , 2006 .

[19]  Xiaotie Deng,et al.  Settling the Complexity of Two-Player Nash Equilibrium , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[20]  Michael P. Wellman,et al.  Empirical mechanism design: methods, with application to a supply-chain scenario , 2006, EC '06.

[21]  Vincent Conitzer,et al.  Self-interested automated mechanism design and implications for optimal combinatorial auctions , 2004, EC '04.

[22]  Paul W. Goldberg,et al.  The complexity of computing a Nash equilibrium , 2006, STOC '06.

[23]  T. W. Ross,et al.  Cooperation without Reputation: Experimental Evidence from Prisoner's Dilemma Games , 1996 .

[24]  Shou-De Lin,et al.  Designing the Market Game for a Trading Agent Competition , 2001, IEEE Internet Comput..

[25]  David M. Kreps,et al.  On the Robustness of Equilibrium Refinements , 1988 .

[26]  Xiaoquan Zhang,et al.  Dynamic price competition on the internet: advertising auctions , 2007, EC '07.

[27]  Jean-Francois Richard,et al.  Empirical Game Theoretic Models: Computational Issues , 2000 .

[28]  Yevgeniy Vorobeychik,et al.  Equilibrium analysis of dynamic bidding in sponsored search auctions , 2007, Int. J. Electron. Bus..

[29]  R. McKelvey,et al.  Computation of equilibria in finite games , 1996 .

[30]  David M. Pennock,et al.  Revenue analysis of a family of ranking rules for keyword auctions , 2007, EC '07.

[31]  H. Robbins A Stochastic Approximation Method , 1951 .

[32]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[33]  Michael P. Wellman,et al.  Distributed Feedback Control for Decision Making on Supply Chains , 2004, ICAPS.

[34]  Elizabeth Sklar,et al.  Co-Evolution of Auction Mechanisms and Trading Strategies: Towards a Novel Approach to Microeconomic , 2002 .

[35]  Craig Boutilier,et al.  Mechanism Design with Partial Revelation , 2007, IJCAI.

[36]  R. Rosenthal Bargaining rules of thumb , 1993 .

[37]  Yoav Shoham,et al.  Spiteful Bidding in Sealed-Bid Auctions , 2007, IJCAI.

[38]  G. Tesauro,et al.  Analyzing Complex Strategic Interactions in Multi-Agent Systems , 2002 .

[39]  B. Stengel,et al.  COMPUTING EQUILIBRIA FOR TWO-PERSON GAMES , 1996 .

[40]  Ashish Sureka,et al.  Using tabu best-response search to find pure strategy nash equilibria in normal form games , 2005, AAMAS '05.

[41]  Pierre L'Ecuyer,et al.  An overview of derivative estimation , 1991, 1991 Winter Simulation Conference Proceedings..

[42]  Dave Cliff,et al.  Evolution of Market Mechanism Through a Continuous Space of Auciton-types II: Two-sided Auction Mechanisms Evolve in Response to Market Shocks , 2002, International Conference on Internet Computing.

[43]  Vincent Conitzer,et al.  Mixed-Integer Programming Methods for Finding Nash Equilibria , 2005, AAAI.

[44]  D. Ruppert The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[45]  E. Maasland,et al.  Auction Theory , 2021, Springer Texts in Business and Economics.

[46]  Norman M. Sadeh,et al.  The supply chain trading agent competition , 2005, Electron. Commer. Res. Appl..

[47]  Roger B. Myerson,et al.  Optimal Auction Design , 1981, Math. Oper. Res..

[48]  Yoav Shoham,et al.  Simple search methods for finding a Nash equilibrium , 2004, Games Econ. Behav..

[49]  Vincent Conitzer,et al.  Incremental Mechanism Design , 2007, IJCAI.

[50]  Michael P. Wellman,et al.  Learning payoff functions in infinite games , 2005, Machine Learning.

[51]  Rajarshi Das,et al.  Choosing Samples to Compute Heuristic-Strategy Nash Equilibrium , 2003, AMEC.

[52]  Lawrence M. Ausubel,et al.  Demand Reduction and Inefficiency in Multi-Unit Auctions , 2014 .

[53]  John O. Ledyard,et al.  Optimal combinatoric auctions with single-minded bidders , 2007, EC '07.

[54]  P. Cramton Simultaneous Ascending Auctions , 2004 .

[55]  Robert Wilson,et al.  A global Newton method to compute Nash equilibria , 2003, J. Econ. Theory.

[56]  Michael P. Wellman,et al.  Iterated Weaker-than-Weak Dominance , 2007, IJCAI.

[57]  Yoav Shoham,et al.  Combinatorial Auctions , 2005, Encyclopedia of Wireless Networks.

[58]  Doina Precup,et al.  Redagent: winner of TAC SCM 2003 , 2004, SECO.

[59]  Andrew W. Moore,et al.  Locally Weighted Learning , 1997, Artificial Intelligence Review.

[60]  Robert L. Smith,et al.  Adaptive search with stochastic acceptance probabilities for global optimization , 2008, Oper. Res. Lett..

[61]  R. McKelvey,et al.  Quantal Response Equilibria for Normal Form Games , 1995 .

[62]  D. Wolpert Predictive Game Theory , 2005 .

[63]  Vincent Conitzer,et al.  An algorithm for automatically designing deterministic mechanisms without payments , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[64]  Michael P. Wellman,et al.  STRATEGIC INTERACTIONS IN A SUPPLY CHAIN GAME , 2005, Comput. Intell..

[65]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[66]  Dave Cliff,et al.  Evolution of market mechanism through a continuous space of auction-types , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[67]  Michael Carl Tschantz,et al.  A stochastic programming approach to scheduling in TAC SCM , 2004, EC '04.

[68]  Elizabeth Sklar,et al.  Using genetic programming to optimise pricing rules for a double auction market , 2010 .

[69]  E. Damme,et al.  Non-Cooperative Games , 2000 .

[70]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[71]  E. H. Clarke Multipart pricing of public goods , 1971 .

[72]  S. Chan ON USING RECURSIVE LEAST SQUARES IN SAMPLE-PATH OPTIMIZATION OF DISCRETE EVENT SYSTEMS , 1995 .

[73]  M.C. Fu,et al.  Simulation optimization , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[74]  Michael P. Wellman,et al.  Generating trading agent strategies: Analytic and empirical methods for infinite and large games , 2005 .

[75]  Paul R. Milgrom,et al.  Rationalizability, Learning, and Equilibrium in Games with Strategic Complementarities , 1990 .

[76]  Kevin Leyton-Brown,et al.  Computing Nash Equilibria of Action-Graph Games , 2004, UAI.

[77]  Sandro Ridella,et al.  Minimizing multimodal functions of continuous variables with the “simulated annealing” algorithmCorrigenda for this article is available here , 1987, TOMS.

[78]  Michael P. Wellman,et al.  Self-Confirming Price Prediction for Bidding in Simultaneous Ascending Auctions , 2005, UAI.

[79]  Claire Mathieu,et al.  Greedy bidding strategies for keyword auctions , 2007, EC '07.

[80]  David M. Kreps,et al.  Game Theory and Economic Modelling , 1992 .

[81]  E. H. Clarke Incentives in public decision-making , 1980 .

[82]  Michael P. Wellman,et al.  An analysis of the 2004 supply chain management trading agent competition , 2005, NAFIPS 2005 - 2005 Annual Meeting of the North American Fuzzy Information Processing Society.

[83]  Michael P. Wellman,et al.  Computing approximate bayes-nash equilibria in tree-games of incomplete information , 2004, EC '04.

[84]  Gerhard Weiß,et al.  Antisocial Agents and Vickrey Auctions , 2001, ATAL.

[85]  Robert Wilson,et al.  Structure theorems for game trees , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[86]  Vincent Conitzer,et al.  Complexity of Mechanism Design , 2002, UAI.

[87]  Michael P. Wellman,et al.  Computing Best-Response Strategies in Infinite Games of Incomplete Information , 2004, UAI.

[88]  Vincent Conitzer,et al.  New complexity results about Nash equilibria , 2008, Games Econ. Behav..

[89]  Michael P. Wellman,et al.  Searching for approximate equilibria in empirical games , 2008, AAMAS.

[90]  J. Mertens,et al.  ON THE STRATEGIC STABILITY OF EQUILIBRIA , 1986 .

[91]  P. Cramton,et al.  Dissolving a Partnership Efficiently , 1985 .

[92]  Mark Fleischer Simulated annealing: past, present, and future , 1995, WSC '95.

[93]  Yi-Ping Chang,et al.  GENERALIZED CONFIDENCE INTERVALS FOR THE LARGEST VALUE OF SOME FUNCTIONS OF PARAMETERS UNDER NORMALITY , 2000 .

[94]  Theodore L. Turocy A dynamic homotopy interpretation of the logistic quantal response equilibrium correspondence , 2005, Games Econ. Behav..

[95]  Kevin Leyton-Brown,et al.  A Polynomial-Time Algorithm for Action Graph Games , 2006, AAAI.

[96]  Daphne Koller,et al.  Multi-Agent Influence Diagrams for Representing and Solving Games , 2001, IJCAI.

[97]  Sébastien Lahaie,et al.  An analysis of alternative slot auction designs for sponsored search , 2006, EC '06.

[98]  D. Friedman EVOLUTIONARY GAMES IN ECONOMICS , 1991 .

[99]  Vincent Conitzer,et al.  Automated Design of Multistage Mechanisms , 2007, IJCAI.

[100]  R. McKelvey A Liapunov Function for Nash Equilibria , 1998 .

[101]  R. Selten,et al.  Experimental Sealed Bid First Price Auctions with Directly Observed Bid Functions , 1994 .

[102]  Michael P. Wellman,et al.  Stochastic Search Methods for Nash Equilibrium Approximation in Simulation-based Games , 2022 .

[103]  Daniel M. Reeves,et al.  Notes on Equilibria in Symmetric Games , 2004 .

[104]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[105]  Eitan Zemel,et al.  Nash and correlated equilibria: Some complexity considerations , 1989 .

[106]  Herbert E. Scarf,et al.  The Approximation of Fixed Points of a Continuous Mapping , 1967 .

[107]  Gül Gürkan,et al.  Sample-path optimization in simulation , 1994, Proceedings of Winter Simulation Conference.

[108]  Michael P. Wellman,et al.  The 2001 trading agent competition , 2002, Electron. Mark..

[109]  M. Hirsch,et al.  On Algorithms for Solving f(x)=0 , 1979 .

[110]  Michael P. Wellman,et al.  Constraint satisfaction algorithms for graphical games , 2007, AAMAS '07.

[111]  Peter Stone,et al.  Adaptive mechanism design: a metalearning approach , 2006, ICEC '06.

[112]  Craig Boutilier,et al.  Partial Revelation Automated Mechanism Design , 2007, AAAI.

[113]  Michael P. Wellman,et al.  Exploring bidding strategies for market-based scheduling , 2003, EC '03.

[114]  Michael P. Wellman,et al.  Selecting strategies using empirical game models: an experimental analysis of meta-strategies , 2008, AAMAS.

[115]  Stuart J. Russell,et al.  Principles of Metareasoning , 1989, Artif. Intell..

[116]  Patrick Siarry,et al.  Enhanced simulated annealing for globally minimizing functions of many-continuous variables , 1997, TOMS.

[117]  Ennio Stacchetti,et al.  A Bound on the Proportion of Pure Strategy Equilibria in Generic Games , 1993, Math. Oper. Res..

[118]  D. Luenberger Optimization by Vector Space Methods , 1968 .

[119]  Yoav Shoham,et al.  Run the GAMUT: a comprehensive approach to evaluating game-theoretic algorithms , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[120]  David C. Parkes,et al.  Passive verification of the strategyproofness of mechanisms in open environments , 2006, ICEC '06.

[121]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[122]  Andrew McLennan,et al.  Gambit: Software Tools for Game Theory , 2006 .

[123]  Paul R. Milgrom,et al.  Putting Auction Theory to Work: The Simultaneous Ascending Auction , 1999, Journal of Political Economy.

[124]  William Vickrey,et al.  Counterspeculation, Auctions, And Competitive Sealed Tenders , 1961 .

[125]  David Pearce Rationalizable Strategic Behavior and the Problem of Perfection , 1984 .

[126]  Arkadi Nemirovski,et al.  Robust optimization – methodology and applications , 2002, Math. Program..

[127]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[128]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[129]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[130]  H. Kuk On equilibrium points in bimatrix games , 1996 .

[131]  Peter Stone,et al.  TacTex-03: a supply chain management agent , 2004, SECO.