Strategy Transfer Learning via Parametric Deviation Sets

Accurately reasoning about agents’ actions in strategic settings is a challenging artificial intelligence task. Many difficulties in learning to perform such reasoning arise due to the uncertainty in both agents’ motives and the strategic games being played. In this paper, we address the problem of learning from observations of the agents’ behavior in some games to predict play in different, but related, games. We introduce a deviation-based strategy prediction approach that, by also using game outcome features to describe different games in a common language, enables generalized strategy learning.

[1]  J. Nash.  NON-COOPERATIVE GAMES , 1951, Classics in Game Theory.

[2]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[3]  R. Aumann.  Subjectivity and Correlation in Randomized Strategies , 1974 .

[4]  Richard S. Varga,et al.  Proof of Theorem 5 , 1983 .

[5]  W. Arthur.  Inductive Reasoning and Bounded Rationality , 1994 .

[6]  R. McKelvey,et al.  Quantal Response Equilibria for Normal Form Games , 1995 .

[7]  Shai Halevi,et al.  A Cryptographic Solution to a Game Theoretic Problem , 2000, CRYPTO.

[8]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[9]  J. Kagel,et al.  Lessons Learned: Generalizing Learning Across Games , 2003 .

[10]  John C. Harsanyi,et al.  Games with Incomplete Information Played by "Bayesian" Players, I-III: Part I. The Basic Model& , 2004, Manag. Sci..

[11]  A. Dawid,et al.  Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory , 2004, math/0410076.

[12]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[13]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[14]  Moshe Tennenholtz,et al.  Learning in One-Shot Strategic Form Games , 2006, ECML.

[15]  Luis E. Ortiz,et al.  Maximum Entropy Correlated Equilibria , 2007, AISTATS.

[16]  Ashwin Ram,et al.  Transfer Learning in Real-Time Strategy Games Using Hybrid CBR/RL , 2007, IJCAI.

[17]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[18]  Geoffrey J. Gordon,et al.  No-regret learning in convex games , 2008, ICML '08.

[19]  Zhou Yang.  Correlated Equilibrium and the Estimation of Discrete Games of Complete Information , 2009 .

[20]  Joseph Y. Halpern,et al.  Iterated Regret Minimization: A New Solution Concept , 2009, IJCAI.

[21]  H. Gintis.  The Bounds of Reason: Game Theory and the Unification of the Behavioral Sciences , 2014 .

[22]  Kevin Leyton-Brown,et al.  Beyond equilibrium: predicting human behaviour in normal form games , 2010, AAAI.

[23]  Anind K. Dey,et al.  Modeling Interaction via the Principle of Maximum Causal Entropy , 2010, ICML.

[24]  Kevin Waugh,et al.  Computational Rationalization: The Inverse Equilibrium Problem , 2011, ICML.