Effective Approximations for Planning with Spatially Distributed Tasks

Planning in cooperative multiagent systems can be neatly formalized using Multi-Agent MDPs, but solving these models is computationally costly. This paper introduces a sub-class of problems called spatial task allocation problems (SPATAPS) that model problems in which a team of agents has to service a dynamically changing set of tasks that is spatially distributed in the environment. We propose to tackle SPATAPS using online, distributed planning by combining subjective agent approximations with restriction of attention to current tasks in the world. An empirical evaluation shows that the combination of both strategies allows to scale to very large problems, while providing near-optimal solutions.

[1]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[2]  Arnaud Doniec,et al.  Scaling Up Decentralized MDPs Through Heuristic Search , 2012, UAI.

[3]  Laurent Jeanpierre,et al.  Coordinated Multi-Robot Exploration Under Communication Constraints Using Decentralized Markov Decision Processes , 2012, AAAI.

[4]  Edmund H. Durfee,et al.  A decision-theoretic characterization of organizational influences , 2012, AAMAS.

[5]  Yifeng Zeng,et al.  Graphical models for interactive POMDPs: representations and solutions , 2009, Autonomous Agents and Multi-Agent Systems.

[6]  Manuela M. Veloso,et al.  Learning of coordination: exploiting sparse interactions in multiagent systems , 2009, AAMAS.

[7]  Craig Boutilier,et al.  Planning, Learning and Coordination in Multiagent Decision Processes , 1996, TARK.

[8]  Edmund H. Durfee,et al.  Resource-Driven Mission-Phasing Techniques for Constrained Agents in Stochastic Environments , 2010, J. Artif. Intell. Res..

[9]  Peter Vrancx,et al.  Learning multi-agent state space representations , 2010, AAMAS.

[10]  Neil Immerman,et al.  The Complexity of Decentralized Control of Markov Decision Processes , 2000, UAI.

[11]  P. J. Gmytrasiewicz,et al.  A Framework for Sequential Planning in Multi-Agent Settings , 2005, AI&M.

[12]  Francisco S. Melo,et al.  Interaction-driven Markov games for decentralized multiagent planning under uncertainty , 2008, AAMAS.

[13]  Nikos A. Vlassis,et al.  Collaborative Multiagent Reinforcement Learning by Payoff Propagation , 2006, J. Mach. Learn. Res..

[14]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[15]  Jesse Hoey,et al.  SPUDD: Stochastic Planning using Decision Diagrams , 1999, UAI.

[16]  Andrew W. Moore,et al.  Distributed Value Functions , 1999, ICML.