Robust Decision Making under Strategic Uncertainty in Multiagent Environments

We introduce the notion of strategic uncertainty for boundedly rational, non-myopic agents as an analog to the equilibrium selection problem in classical game theory. We then motivate the need for and feasibility of addressing strategic uncertainty and present an algorithm that produces decisions that are robust to it. Finally, we show how agents' rationality levels and planning horizons alter the robustness of their decisions.

[1]  Edmund H. Durfee,et al.  Predicting the Expected Behavior of Agents that Learn About Agents: The CLRI Framework , 2004, Autonomous Agents and Multi-Agent Systems.

[2]  H. Van Dyke Parunak,et al.  Concurrent Modeling of Alternative Worlds with Polyagents , 2006, MABS.

[3]  John B. Gilmer,et al.  Support to decision makers: the use of recursive simulation to support decisionmaking , 2003, WSC '03.

[4]  H. Barger.  The General Theory of Employment, Interest and Money , 1936, Nature.

[5]  Miguel A. Costa-Gomes,et al.  Comparing Models of Strategic Thinking in Van Huyck, Battalio, and Beil’s Coordination Games , 2009 .

[6]  J. B. Jr. Gilmer,et al.  The use of recursive simulation to support decisionmaking , 2003, Proceedings of the 2003 Winter Simulation Conference, 2003..

[7]  J. Rosenhead,et al.  Robustness and Optimality as Criteria for Strategic Decisions , 1972 .

[8]  J. Keynes,et al.  The General Theory of Employment, Interest and Money. , 1936 .

[9]  Leslie Pack Kaelbling,et al.  Multi-Agent Filtering with Infinitely Nested Beliefs , 2008, NIPS.

[10]  Colin Camerer,et al.  A Cognitive Hierarchy Model of Games , 2004 .

[11]  Akira Hayashi,et al.  A multiagent reinforcement learning algorithm using extended optimal response , 2002, AAMAS '02.

[12]  Michael P. Wellman,et al.  Nash Q-Learning for General-Sum Stochastic Games , 2003, J. Mach. Learn. Res..

[13]  Edmund H. Durfee,et al.  Recursive Agent Modeling Using Limited Rationality , 1995, ICMAS.

[14]  Pieter Spronck,et al.  Monte-Carlo Tree Search: A New Framework for Game AI , 2008, AIIDE.

[15]  Shlomo Zilberstein,et al.  Memory-Bounded Dynamic Programming for DEC-POMDPs , 2007, IJCAI.

[16]  R. McKelvey,et al.  Quantal Response Equilibria for Extensive Form Games , 1998 .

[17]  Michael P. Wellman,et al.  Learning about other agents in a dynamic multiagent system , 2001, Cognitive Systems Research.

[18]  John B. Gilmer,et al.  Issues in event analysis for recursive simulation , 2005, Proceedings of the Winter Simulation Conference, 2005..

[19]  J. Huyck,et al.  Strategic Uncertainty, Equilibrium Selection, and Coordination Failure in Average Opinion Games , 1991 .

[20]  Robert Givan,et al.  FF-Replan: A Baseline for Probabilistic Planning , 2007, ICAPS.

[21]  Dimitris Bertsimas,et al.  Robust game theory , 2006, Math. Program..

[22]  D. Stahl,et al.  Experimental evidence on players' models of other players , 1994 .

[23]  David V. Pynadath,et al.  Towards Computing Optimal Policies for Decentralized POMDPs , 2002 .

[24]  Oliver Alfred Gross,et al.  A Continuous Colonel Blotto Game , 1950 .

[25]  Sergiu Hart,et al.  Discrete Colonel Blotto and General Lotto games , 2008, Int. J. Game Theory.

[26]  Ariel Rubinstein,et al.  Colonel Blotto’s Top Secret Files , 2010 .

[27]  Scott E. Page,et al.  General Blotto: games of allocative strategic mismatch , 2009 .

[28]  Sanguk Noh,et al.  Bayesian Update of Recursive Agent Models , 2004, User Modeling and User-Adapted Interaction.

[29]  M. Latek,et al.  Social Learning and Pricing Obfuscation , 2009 .

[30]  Dale O. Stahl,et al.  Level-n Bounded Rationality on a Level Playing Field of Sequential Games , 2004 .

[31]  Shlomo Zilberstein,et al.  Dynamic Programming for Partially Observable Stochastic Games , 2004, AAAI.

[32]  R. McKelvey,et al.  Quantal Response Equilibria for Normal Form Games , 1995 .

[33]  Maciej M Latek,et al.  Optimal Blends of History and Intelligence for Robust Antiterrorism Policy , 2011 .

[34]  V. Crawford,et al.  Learning How to Cooperate: Optimal Play in Repeated Coordination Games , 1990 .

[35]  Emerson M. S. Niou,et al.  A Game-Theoretic Interpretation of Sun Tzu's , 1994 .

[36]  Robert L. Axtell,et al.  WHY AGENTS? ON THE VARIED MOTIVATIONS FOR AGENT COMPUTING IN THE SOCIAL SCIENCES , 2000 .

[37]  Prashant Doshi,et al.  Exact solutions of interactive POMDPs using behavioral equivalence , 2006, AAMAS '06.

[38]  David Carmel,et al.  Incorporating Opponent Models into Adversary Search , 1996, AAAI/IAAI, Vol. 1.

[39]  Bogumil Kaminski,et al.  The Influence of Call Graph Topology on the Dynamics of Telecommunication Markets , 2010, KES-AMSTA.

[40]  Sarit Kraus,et al.  Robust solutions to Stackelberg games: Addressing bounded rationality and limited observations in human cognition , 2010, Artif. Intell..

[41]  Bogumil Kaminski,et al.  Bounded rationality via recursion , 2009, AAMAS.