A Hybrid Cooperative Behavior Learning Method for a Rule-Based Shout-Ahead Architecture

We present an agent architecture and a hybrid behavior learning method for it that allows the use of communicated intentions of other agents to create agents that are able to cooperate with various configurations of other agents in fulfilling a task. Our shout-ahead architecture is based on two rule sets, one making decisions without communicated intentions and one with these intentions. Reinforcement learning is used to determine in a particular situation which set is responsible for the final decision. Evolutionary learning is used to learn these rules. Our application of this approach to learning behaviors for units in a computer game shows that the use of shout-ahead substantially improves the quality of the learned behavior compared to agents not using shout-ahead.

[1]  Risto Miikkulainen,et al.  COOPERATIVE COEVOLUTION OF MULTI-AGENT SYSTEMS , 2001 .

[2]  Randall Davis,et al.  Negotiation as a Metaphor for Distributed Problem Solving , 1988, Artif. Intell..

[3]  Matthias Fuchs,et al.  Experiments in learning prototypical situations for variants of the pursuit game , 1999 .

[4]  Nicolas Sabouret,et al.  Reinforcement Learning of Communication in a Multi-agent Context , 2011, 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology.

[5]  Marc J. V. Ponsen,et al.  Improving Adaptive Game Ai with Evolutionary Learning , 2004 .

[6]  Sean Luke,et al.  Cooperative Multi-Agent Learning: The State of the Art , 2005, Autonomous Agents and Multi-Agent Systems.

[7]  Dipti Srinivasan,et al.  Cooperative, hybrid agent architecture for real-time traffic signal control , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[8]  D. Vengerov,et al.  Learning, Cooperation, and Coordination in Multi-Agent Systems , 2000 .

[9]  Jörg Denzinger,et al.  On cooperation between evolutionary algorithms and other search paradigms , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[10]  Regina Barzilay,et al.  Learning to Win by Reading Manuals in a Monte-Carlo Framework , 2011, ACL.

[11]  Randall Davis,et al.  Negotiation as a Metaphor for Distributed Problem Solving , 1988, Artificial Intelligence.

[12]  Victor R. Lesser,et al.  Functionally Accurate, Cooperative Distributed Systems , 1988, IEEE Transactions on Systems, Man, and Cybernetics.

[13]  Alain Dutech,et al.  Cooperation through communication in decentralized Markov games , 2004 .

[14]  Claudia V. Goldman,et al.  Optimizing information exchange in cooperative multi-agent systems , 2003, AAMAS '03.

[15]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .