Learning payoff functions in infinite games

We consider a class of games with real-valued strategies and payoff information available only in the form of data from a given sample of strategy profiles. Solving such games with respect to the underlying strategy space requires generalizing from the data to a complete payoff-function representation. We address payoff-function learning as a standard regression problem, with provision for capturing known structure (e.g., symmetry) in the multiagent environment. To measure learning performance, we consider the relative utility of prescribed strategies, rather than the accuracy of payoff functions per se. We demonstrate our approach and evaluate its effectiveness on two examples: a two-player version of the first-price sealed-bid auction (with known analytical form), and a five-player market-based scheduling game (with no known solution). Additionally, we explore the efficacy of using relative utility of strategies as a target of supervised learning and as a learning model selector. Our experiments demonstrate its effectiveness in the former case, though not in the latter.

[1]  W. Browder,et al.  Annals of Mathematics , 1889 .

[2]  William Vickrey,et al.  Counterspeculation, Auctions, And Competitive Sealed Tenders , 1961 .

[3]  Nils J. Nilsson,et al.  Artificial Intelligence , 1974, IFIP Congress.

[4]  David M. Kreps,et al.  Game Theory and Economic Modelling , 1992 .

[5]  D. Friedman EVOLUTIONARY GAMES IN ECONOMICS , 1991 .

[6]  R. Selten,et al.  Experimental Sealed Bid First Price Auctions with Directly Observed Bid Functions , 1994 .

[7]  E. Balder Remarks on Nash equilibria for games with additively coupled payoffs , 1995 .

[8]  A. Mas-Colell,et al.  Microeconomic Theory , 1995 .

[9]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[10]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[11]  B. Schölkopf,et al.  Advances in kernel methods: support vector learning , 1999 .

[12]  Eric van Damme,et al.  Non-Cooperative Games , 2000 .

[13]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[14]  J. Wyatt Decision support systems. , 2000, Journal of the Royal Society of Medicine.

[15]  Paul R. Milgrom,et al.  Putting Auction Theory to Work: The Simultaneous Ascending Auction , 1999, Journal of Political Economy.

[16]  Ronen I. Brafman,et al.  Efficient learning equilibrium , 2004, Artificial Intelligence.

[17]  Daniel M. Reeves Generating trading agent strategies , 2002, AAAI/IAAI.

[18]  Constance de Koning,et al.  Editors , 2003, Annals of Emergency Medicine.

[19]  Yoav Shoham,et al.  Multi-Agent Reinforcement Learning:a critical survey , 2003 .

[20]  Michael P. Wellman,et al.  Exploring bidding strategies for market-based scheduling , 2003, EC '03.

[21]  Ronen I. Brafman,et al.  Learning to Coordinate Efficiently: A Model-based Approach , 2003, J. Artif. Intell. Res..

[22]  P. Cramton Simultaneous Ascending Auctions , 2004 .

[23]  Andrew W. Moore,et al.  Locally Weighted Learning , 1997, Artificial Intelligence Review.

[24]  Michael P. Wellman,et al.  Computing Best-Response Strategies in Infinite Games of Incomplete Information , 2004, UAI.

[25]  Daniel M. Reeves,et al.  Notes on Equilibria in Symmetric Games , 2004 .

[26]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[27]  Michael P. Wellman,et al.  Generating trading agent strategies: Analytic and empirical methods for infinite and large games , 2005 .

[28]  E. Maasland,et al.  Auction Theory , 2021, Springer Texts in Business and Economics.

[29]  Yoav Shoham,et al.  If multi-agent learning is the answer, what is the question? , 2007, Artif. Intell..

[30]  Sherwood C. Frey,et al.  Auctions , 2009, SSRN Electronic Journal.

[31]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.