Task allocation learning in a multiagent environment: Application to the RoboCupRescue simulation

Coordinating agents in a complex environment is a hard problem, but it can become even harder when certain characteristics of the tasks, like the required number of agents, are unknown. In these settings, agents not only have to coordinate themselves on the different tasks, but they also have to learn how many agents are required for each task. To contribute to this problem, we present in this paper a selective perception reinforcement learning algorithm which enables agents to learn the required number of agents that should coordinate their efforts on a given task. Even though there are continuous variables in the task description, agents in our algorithm are able to learn their expected reward according to the task description and the number of agents. The results, obtained in the RoboCupRescue simulation environment, show an improvement in the agents overall performance.

[1]  Manuela M. Veloso,et al.  TTree: Tree-Based State Generalization with Temporally Abstract Actions , 2002, SARA.

[2]  Lovekesh Vig,et al.  Coalition Formation: From Software Agents to Robots , 2007, J. Intell. Robotic Syst..

[3]  Dana Ron,et al.  Learning probabilistic automata with variable memory length , 1994, COLT '94.

[4]  Bernhard Nebel,et al.  Successful Search and Rescue in Simulated Disaster Areas , 2005, RoboCup.

[5]  Barbara Messing,et al.  An Introduction to MultiAgent Systems , 2002, Künstliche Intell..

[6]  J. Ross Quinlan,et al.  Combining Instance-Based and Model-Based Learning , 1993, ICML.

[7]  Andrew McCallum,et al.  Reinforcement learning with selective perception and hidden state , 1996 .

[8]  Claudia V. Goldman,et al.  Self-organization through bottom-up coalition formation , 2003, AAMAS '03.

[9]  Manuela M. Veloso,et al.  TTree: Tree-Based State Generalization with Temporally Abstract Actions , 2002, Adaptive Agents and Multi-Agents Systems.

[10]  Anthony Stentz,et al.  A Free Market Architecture for Distributed Control of a Multirobot System , 2000 .

[11]  Olivier Buffet,et al.  Multi-Agent Systems by Incremental Gradient Reinforcement Learning , 2001, IJCAI.

[12]  Yoav Shoham,et al.  An overview of combinatorial auctions , 2007, SECO.

[13]  Hiroaki Kitano,et al.  RoboCup Rescue: a grand challenge for multi-agent systems , 2000, Proceedings Fourth International Conference on MultiAgent Systems.

[14]  Michael Ströbel,et al.  On Auctions as the Negotiation Paradigm of Electronic Markets , 2000, Electron. Mark..

[15]  Brahim Chaib-draa,et al.  Selective Perception Learning for Tasks Allocation , 2004 .

[16]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[17]  Brahim Chaib-draa,et al.  Multi-attribute Decision Making in a Complex Multiagent Environment Using Reinforcement Learning with Selective Perception , 2004, Canadian Conference on AI.

[18]  Alexis Drogoul,et al.  Interactive Learning of Independent Experts' Criteria for Rescue Simulations , 2009, J. Univers. Comput. Sci..

[19]  Manuela M. Veloso,et al.  Tree Based Discretization for Continuous State Space Reinforcement Learning , 1998, AAAI/IAAI.

[20]  Craig Boutilier,et al.  Bayesian reinforcement learning for coalition formation under uncertainty , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[21]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[22]  Maria L. Gini,et al.  Auctions for task allocation to robots , 2006, IAS.

[23]  Maja J. Mataric,et al.  Multi-robot task allocation: analyzing the complexity and optimality of key architectures , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[24]  Craig Boutilier,et al.  Planning, Learning and Coordination in Multiagent Decision Processes , 1996, TARK.

[25]  Michael Wooldridge,et al.  Introduction to multiagent systems , 2001 .

[26]  Richard Alterman,et al.  Autonomous Agents that Learn to Better Coordinate , 2004, Autonomous Agents and Multi-Agent Systems.

[27]  Andrew W. Moore,et al.  The parti-game algorithm for variable resolution reinforcement learning in multidimensional state-spaces , 2004, Machine Learning.

[28]  Reid G. Smith,et al.  The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver , 1980, IEEE Transactions on Computers.

[29]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[30]  Sarit Kraus,et al.  Methods for Task Allocation via Agent Coalition Formation , 1998, Artif. Intell..

[31]  Michael G. Madden,et al.  Coalition calculation in a dynamic agent environment , 2004, ICML.

[32]  Peter Stone,et al.  Behavior transfer for value-function-based reinforcement learning , 2005, AAMAS '05.