Being Reactive by Exchanging Roles: An Empirical Study

In the multi-agent community, the need for social deliberation appears contradictory with the need for reactivity. In this paper, we try to show that we can draw the benefits of both being reactive and being socially organized thanks to what we call "social reactivity".In order to defend this claim, we describe a simulation experiment in which several sheepdog agents have to coordinate their effort to drive a flock of ducks towards a goal area. We implement reactive controllers for agents in the Classifier Systems formalism and we compare the performance of purely reactive, solipsistic agents which are coordinated implicitly with the performance of agents using roles. We show that our role-based agents perform better than the solipsistic ones, but because of constraints on the roles of the agents, the solipsistic controllers are more robust and more opportunistic. Then we show that, by exchanging reactively their roles, a process which can be seen as implementing a form of social deliberation, role-based agents finally outperform the solipsistic ones. Since designing by hand the rules for exchanging the roles proved difficult, we conclude by advocating the necessity of tackling the problem of letting the agents learn their own role exchange processes.

[1]  Olivier Sigaud,et al.  YACS: a new learning classifier system using anticipation , 2002, Soft Comput..

[2]  Abdel-Illah Mouaddib,et al.  Deliberation Levels in Theoretic-Decision Approaches for Task Allocation in Resource-Bounded Agents , 2000, Balancing Reactivity and Social Deliberation in Multi-Agent Systems.

[3]  Minoru Asada,et al.  Cooperative Behavior Acquisition for Mobile Robots in Dynamically Changing Real Worlds Via Vision-Based Reinforcement Learning and Development , 1999, Artif. Intell..

[4]  Rick L. Riolo,et al.  Lookahead planning and latent learning in a classifier system , 1991 .

[5]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[6]  Stewart W. Wilson Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[7]  Stewart W. Wilson ZCS: A Zeroth Level Classifier System , 1994, Evolutionary Computation.

[8]  Olivier Sigaud,et al.  YACS: Combining Dynamic Programming with Generalization in Classifier Systems , 2000, IWLCS.

[9]  Milind Tambe,et al.  Building Agent Teams Using an Explicit Teamwork Model and Learning , 1999, Artif. Intell..

[10]  R. Bellman Dynamic programming. , 1957, Science.

[11]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[12]  Manuela M. Veloso,et al.  Task Decomposition, Dynamic Role Assignment, and Low-Bandwidth Communication for Real-Time Strategic Teamwork , 1999, Artif. Intell..

[13]  Olivier Sigaud,et al.  Using Classifier Systems as Adaptive Expert Systems for Control , 2000, IWLCS.

[14]  Maja J. Mataric,et al.  Reward Functions for Accelerated Learning , 1994, ICML.

[15]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[16]  Zbigniew Michalewicz,et al.  Evolutionary Computation 2 , 2000 .

[17]  Manuela M. Veloso,et al.  Multiagent Systems: A Survey from a Machine Learning Perspective , 2000, Auton. Robots.

[18]  Andrew W. Moore,et al.  Reinforcement Learning for Cooperating and Communicating Reactive Agents in Electrical Power Grids , 2000, Balancing Reactivity and Social Deliberation in Multi-Agent Systems.

[19]  Ansgar Bredenfeld,et al.  Team Cooperation Using Dual Dynamics , 2000, Balancing Reactivity and Social Deliberation in Multi-Agent Systems.

[20]  Marco Colombetti,et al.  Training Agents to Perform Sequential Behavior , 1994, Adapt. Behav..

[21]  Rodney A. Brooks,et al.  Intelligence Without Reason , 1991, IJCAI.

[22]  Sven Behnke,et al.  A Hierarchy of Reactive Behaviors Handles Complexity , 2000, Balancing Reactivity and Social Deliberation in Multi-Agent Systems.

[23]  Leslie Pack Kaelbling,et al.  An Architecture for Intelligent Reactive Systems , 1987 .

[24]  Maja J. Mataric,et al.  Interaction and intelligent behavior , 1994 .

[25]  Stephen Cameron,et al.  Robot sheepdog project achieves automatic flock control , 1998 .