Practical Strategic Reasoning with Applications in Market Games

Strategic reasoning is part of our everyday lives: we negotiate prices, bid in auctions, write contracts, and play games. We choose actions in these scenarios based on our preferences, and our beliefs about preferences of the other participants. Game theory provides a rich mathematical framework through which we can reason about the influence of these preferences. Clever abstractions allow us to predict the outcome of complex agent interactions, however, as the scenarios we model increase in complexity, the abstractions we use to enable classical game-theoretic analysis lose fidelity. In empirical game-theoretic analysis, we construct game models using empirical sources of knowledge—such as high-fidelity simulation. However, utilizing empirical knowledge introduces a host of different computational and statistical problems. I investigate five main research problems that focus on efficient selection, estimation, and analysis of empirical game models. I introduce a flexible modeling approach, where we may construct multiple game-theoretic models from the same set of observations. I propose a principled methodology for comparing empirical game models and a family of algorithms that select a model from a set of candidates. I develop algorithms for normal-form games that efficiently identify formations—sets of strategies that are closed under a (correlated) best-response correspondence. This aids in problems, such as finding Nash equilibria, that are key to analysis but hard to solve. I investigate policies for sequentially determining profiles to simulate, when constrained by a budget for simulation. Efficient policies allow modelers to analyze complex scenarios by evaluating a subset of the profiles. The policies I introduce outperform the existing policies in experiments. I establish a principled methodology for evaluating strategies given an empirical game model. I employ this methodology in two case studies of market scenarios: first, a case study in supply chain management from the perspective of a strategy designer; then, a case study in Internet ad auctions from the perspective of a mechanism designer. As part of the latter analysis, I develop an ad-auctions scenario that captures several key strategic issues in this domain for the first time.

[1]  John Langford,et al.  Maintaining Equilibria During Exploration in Sponsored Search Auctions , 2007, WINE.

[2]  David P. Williamson,et al.  An adaptive algorithm for selecting profitable keywords for search-based advertising services , 2006, EC '06.

[3]  R. Vohra,et al.  Algorithmic Game Theory: Sponsored Search Auctions , 2007 .

[4]  Benjamin Edelman,et al.  Optimal Auction Design in a Multi-unit Environment : The Case of Sponsored Search Auctions , 2007 .

[5]  Nicholas R. Jennings,et al.  Designing a successful trading agent for supply chain management , 2006, AAMAS '06.

[6]  Maria Gini,et al.  Software architecture of the MinneTAC supply-chain trading agent , 2008 .

[7]  Peter Stone,et al.  The 2007 TAC SCM Prediction Challenge , 2008, AMEC/TADA.

[8]  Hemant K. Bhargava,et al.  Implementing Sponsored Search in Web Search Engines: Computational Evaluation of Alternative Mechanisms , 2007, INFORMS J. Comput..

[9]  Vibhanshu Abhishek,et al.  Keyword generation for search engine advertising using semantic similarity between terms , 2007, ICEC.

[10]  Brendan Kitts,et al.  Optimal Bidding on Keyword Auctions , 2004, Electron. Mark..

[11]  Joseph Y. Halpern,et al.  Iterated Regret Minimization: A New Solution Concept , 2009, IJCAI.

[12]  Amanda Spink,et al.  Determining the informational, navigational, and transactional intent of Web queries , 2008, Inf. Process. Manag..

[13]  M. Stone Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[14]  Norman M. Sadeh,et al.  The supply chain trading agent competition , 2005, Electron. Commer. Res. Appl..

[15]  James R. Wilson Variance Reduction Techniques for Digital Simulation , 1984 .

[16]  Yevgeniy Vorobeychik,et al.  Equilibrium analysis of dynamic bidding in sponsored search auctions , 2007, Int. J. Electron. Bus..

[17]  Michael P. Wellman,et al.  Empirical Game-Theoretic Analysis of Chaturanga , 2006 .

[18]  Ashish Sureka,et al.  Using tabu best-response search to find pure strategy nash equilibria in normal form games , 2005, AAMAS '05.

[19]  Gordan Jezic,et al.  The CrocodileAgent: Research for Efficient Agent-Based Cross-Enterprise Processes , 2006, OTM Workshops.

[20]  Daniel C. Fain,et al.  Sponsored search: A brief history , 2006 .

[21]  Victor Naroditskiy,et al.  An Algorithm for Stochastic Multiple-Choice Knapsack Problem and Keywords Bidding , 2008 .

[22]  Mark Voorneveld Persistent retracts and preparation , 2005, Games Econ. Behav..

[23]  Jon Feldman,et al.  Algorithmic Methods for Sponsored Search Advertising , 2008, ArXiv.

[24]  Yoav Shoham,et al.  Run the GAMUT: a comprehensive approach to evaluating game-theoretic algorithms , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[25]  G. Tesauro,et al.  Analyzing Complex Strategic Interactions in Multi-Agent Systems , 2002 .

[26]  ILLUSTRATION OF THE SAMPLE SPACE DEFINITION OF SIMULATION AND VARIANCE REDUCTION. , 1985 .

[27]  Michael P. Wellman,et al.  Approximate Strategic Reasoning through Hierarchical Reduction of Large Symmetric Games , 2005, AAAI.

[28]  Rajarshi Das,et al.  Choosing Samples to Compute Heuristic-Strategy Nash Equilibrium , 2003, AMEC.

[29]  John Collins,et al.  The Supply Chain Management Game for the 2007 Trading Agent Competition , 2004 .

[30]  Yoav Shoham,et al.  Simple search methods for finding a Nash equilibrium , 2004, Games Econ. Behav..

[31]  N. Metropolis THE BEGINNING of the MONTE CARLO METHOD , 2022 .

[32]  Michael P. Wellman,et al.  Forecasting market prices in a supply chain game , 2007, AAMAS '07.

[33]  Ashish Goel,et al.  Truthful auctions for pricing search keywords , 2006, EC '06.

[34]  Felix A. Fischer,et al.  Computational aspects of Shapley's saddles , 2009, AAMAS.

[35]  Maria L. Gini,et al.  Efficient Statistical Methods for Evaluating Trading Agent Performance , 2007, AAAI.

[36]  J. Neumann,et al.  Theory of Games and Economic Behavior. , 1945 .

[37]  Michael P. Wellman,et al.  Knowledge Combination in Graphical Multiagent Models , 2008, UAI.

[38]  Louis Raymond,et al.  The advantages of electronic data interchange , 1992, DATB.

[39]  E. Rowland Theory of Games and Economic Behavior , 1946, Nature.

[40]  Stephen S. Lavenberg,et al.  Statistical Results on Control Variables with Application to Queueing Network Simulation , 1982, Oper. Res..

[41]  Arpita Ghosh,et al.  Auctions with Revenue Guarantees for Sponsored Search , 2007, WINE.

[42]  N. Metropolis,et al.  The Monte Carlo method. , 1949 .

[43]  Michael P. Wellman,et al.  Searching for approximate equilibria in empirical games , 2008, AAMAS.

[44]  Pierre L'Ecuyer,et al.  Efficiency improvement and variance reduction , 1994, Proceedings of Winter Simulation Conference.

[45]  Nick Craswell,et al.  An experimental comparison of click position-bias models , 2008, WSDM '08.

[46]  Stephen S. Lavenberg,et al.  A Perspective on the Use of Control Variables to Increase the Efficiency of Monte Carlo Simulations , 1981 .

[47]  J. Hammersley,et al.  Monte Carlo Methods , 1966 .

[48]  Aranyak Mehta,et al.  Online budgeted matching in random input models with applications to Adwords , 2008, SODA '08.

[49]  Michael H. Bowling,et al.  Optimal Unbiased Estimators for Evaluating Agent Performance , 2006, AAAI.

[50]  Glenn Ellison,et al.  Position Auctions with Consumer Search , 2007 .

[51]  Vincent Conitzer,et al.  A Generalized Strategy Eliminability Criterion and Computational Methods for Applying It , 2005, AAAI.

[52]  Vijay Murthi,et al.  Logistic Regression and Collaborative Filtering for Sponsored Search Term Recommendation , 2006 .

[53]  Zoë Abrams,et al.  Ad Auction Design and User Experience , 2007, WINE.

[54]  Michael P. Wellman,et al.  Empirical mechanism design: methods, with application to a supply-chain scenario , 2006, EC '06.

[55]  S. Muthukrishnan,et al.  Stochastic Models for Budget Optimization in Search-Based Advertising , 2006, Algorithmica.

[56]  Peter McBurney,et al.  Characterizing effective auction mechanisms: insights from the 2007 TAC market design competition , 2008, AAMAS.

[57]  Tom E. Yoon,et al.  A DECADE OF SCM LITERATURE: PAST, PRESENT AND FUTURE IMPLICATIONS , 2008 .

[58]  Tamara G. Kolda,et al.  Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods , 2003, SIAM Rev..

[59]  Jean-Francois Richard,et al.  Approximation of Bayesian Nash Equilibrium , 2008 .

[60]  Tuomas Sandholm,et al.  Algorithms for Rationalizability and CURB Sets , 2006, AAAI.

[61]  Peter Stone,et al.  TacTex-05: A Champion Supply Chain Management Agent , 2006, AAAI.

[62]  Michael P. Wellman,et al.  Exploring Large Strategy Spaces in Empirical Game Modeling , 2009 .

[63]  Jon Feldman,et al.  Sponsored Search Auctions with Markovian Users , 2008, WINE.

[64]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[65]  J. Nash Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[66]  Stuart J. Russell,et al.  Principles of Metareasoning , 1989, Artif. Intell..

[67]  A. Gunawardana,et al.  Aggregators and Contextual Effects in Search Ad Markets , 2008 .

[68]  Paul W. Goldberg,et al.  The Complexity of Computing a Nash Equilibrium , 2009, SIAM J. Comput..

[69]  Michael P. Wellman,et al.  STRATEGIC INTERACTIONS IN A SUPPLY CHAIN GAME , 2005, Comput. Intell..

[70]  B. Bernheim Rationalizable Strategic Behavior , 1984 .

[71]  David M. Pennock,et al.  Revenue analysis of a family of ranking rules for keyword auctions , 2007, EC '07.

[72]  Duane Szafron,et al.  Strategy evaluation in extensive games with importance sampling , 2008, ICML '08.

[73]  Claire Mathieu,et al.  Greedy bidding strategies for keyword auctions , 2007, EC '07.

[74]  Michael P. Wellman,et al.  Autonomous bidding agents - strategies and lessons from the trading agent competition , 2007 .

[75]  Norman M. Sadeh,et al.  Factoring games to isolate strategic interactions , 2007, AAMAS '07.

[76]  Yongmin Chen,et al.  Paid Placement: Advertising and Search on the Internet , 2006 .

[77]  Michael P. Wellman Methods for Empirical Game-Theoretic Analysis , 2006, AAAI.

[78]  Jean-Francois Richard,et al.  Empirical Game Theoretic Models: Constrained Equilibrium & Simulation , 1998 .

[79]  Maria Gini,et al.  A survey of agent designs for TAC SCM , 2008, AAAI 2008.

[80]  Anna R. Karlin,et al.  On the Effects of Competing Advertisements in Keyword Auctions , 2008 .

[81]  Michael P. Wellman,et al.  Empirical game-theoretic analysis of the TAC Supply Chain game , 2007, AAMAS '07.

[82]  Brendan Kitts,et al.  A trading agent and simulator for keyword auctions , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[83]  S. Muthukrishnan,et al.  General auction mechanism for search advertising , 2008, WWW '09.

[84]  David Pearce Rationalizable Strategic Behavior and the Problem of Perfection , 1984 .

[85]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[86]  L. J. Savage,et al.  The Foundations of Statistics , 1955 .

[87]  David C. Parkes,et al.  Learning and Solving Many-Player Games through a Cluster-Based Representation , 2008, UAI.

[88]  S. Muthukrishnan,et al.  Stochastic Models for Budget Optimization in Search-Based Advertising , 2007, WINE.

[89]  Adina Magda Florea,et al.  A Dynamic Strategy Agent for Supply Chain Management , 2006, 2006 Eighth International Symposium on Symbolic and Numeric Algorithms for Scientific Computing.

[90]  Vahab S. Mirrokni,et al.  Bid optimization for broad match ad auctions , 2009, WWW '09.

[91]  Jon Feldman,et al.  Position Auctions with Bidder-Specific Minimum Prices , 2008, WINE.

[92]  Anindya Ghose,et al.  Analyzing search engine advertising: firm behavior and cross-selling in electronic markets , 2008, WWW.

[93]  Michael P. Wellman,et al.  Stronger CDA strategies through empirical game-theoretic analysis and reinforcement learning , 2009, AAMAS.

[94]  J. Nash NON-COOPERATIVE GAMES , 1951, Classics in Game Theory.

[95]  Christos H. Papadimitriou,et al.  Three-Player Games Are Hard , 2005, Electron. Colloquium Comput. Complex..

[96]  John C. Harsanyi,et al.  Общая теория выбора равновесия в играх / A General Theory of Equilibrium Selection in Games , 1989 .

[97]  M. J. D. Powell,et al.  Direct search algorithms for optimization calculations , 1998, Acta Numerica.

[98]  J. Weibull,et al.  Strategy subsets closed under rational behavior , 1991 .

[99]  Xiaotie Deng,et al.  Settling the Complexity of Two-Player Nash Equilibrium , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[100]  Yifan Chen,et al.  Advertising keyword suggestion based on concept hierarchy , 2008, WSDM '08.

[101]  David P. Stone An Autonomous Agent for Supply Chain Management , 2007 .

[102]  Amin Saberi,et al.  Multi-unit auctions with unknown supply , 2006, EC '06.

[103]  Norman M. Sadeh,et al.  CMieux: adaptive strategies for competitive supply chain trading , 2006, SIGecom Exch..

[104]  Maria L. Gini To compete or not compete ? Ingredients for a successful competition , 2009 .

[105]  John Collins,et al.  An experiment management framework for TAC SCM agent evaluation , 2009 .

[106]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[107]  Michael P. Wellman,et al.  Mechanism Design Based on Beliefs about Responsive Play ( Position Paper ) , 2006 .

[108]  Huaiyu Zhu On Information and Sufficiency , 1997 .

[109]  Yunhong Zhou,et al.  Vindictive bidding in keyword auctions , 2007, ICEC.

[110]  Jeremy I. Bulow,et al.  Auctions versus Negotiations , 1996 .

[111]  Aranyak Mehta,et al.  AdWords and generalized on-line matching , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[112]  Deeparnab Chakrabarty,et al.  Budget constrained bidding in keyword auctions and online knapsack problems , 2008, WWW.

[113]  Scott Duke Kominers,et al.  Dynamic Position Auctions with Consumer Search , 2008, AAIM.

[114]  Jason Miller,et al.  Controlling a supply chain agent using value-based decomposition , 2006, EC '06.

[115]  Mohammad Mahdian,et al.  A Cascade Model for Externalities in Sponsored Search , 2008, WINE.

[116]  Michael P. Wellman,et al.  Exploring bidding strategies for market-based scheduling , 2003, EC '03.

[117]  R. Rosenthal A class of games possessing pure-strategy Nash equilibria , 1973 .

[118]  Michael P. Wellman,et al.  Searching for Walverine 2005 , 2005, AMEC@AAMAS/TADA@IJCAI.

[119]  Joakim Eriksson,et al.  Evolution of a supply chain management game for the Trading Agent Competition , 2006, AI Commun..

[120]  S. Muthukrishnan,et al.  Internet Ad Auctions: Insights and Directions , 2008, ICALP.

[121]  Pericles A. Mitkas,et al.  A Robust Agent Design for Dynamic SCM Environments , 2006, SETN.

[122]  Michael P. Wellman,et al.  Iterated Weaker-than-Weak Dominance , 2007, IJCAI.

[123]  Michael P. Wellman,et al.  Generating trading agent strategies: Analytic and empirical methods for infinite and large games , 2005 .

[124]  J. Harsanyi Games with Incomplete Information Played by “Bayesian” Players Part II. Bayesian Equilibrium Points , 1968 .

[125]  A. Roth,et al.  Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria , 1998 .

[126]  E. Kalai,et al.  Persistent equilibria in strategic games , 1984 .

[127]  Amin Saberi,et al.  Allocating online advertisement space with unreliable estimates , 2007, EC '07.

[128]  Michael Carl Tschantz,et al.  Botticelli: a supply chain management agent , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..