Multi-agent patrolling with reinforcement learning

Patrolling tasks can be encountered in a variety of real-world domains, ranging from computer network administration and surveillance to computer wargame simulations. It is a complex multi-agent task, which usually requires agents to coordinate their decision-making in order to achieve optimal performance of the group as a whole. In this paper, we show how the patrolling task can be modeled as a reinforcement learning (RL) problem, allowing continuous and automatic adaptation of the agentsý strategies to their environment. We demonstrate that an efficient cooperative behavior can be achieved by using RL methods, such as Q-Learning, to train individual agents. The proposed approach is totally distributed, which makes it computationally efficient. The empirical evaluation proves the effectiveness of our approach, as the results obtained are substantially better than the results available so far on this domain.

[1]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[2]  K. Upton,et al.  A modern approach , 1995 .

[3]  Peter Stone,et al.  Layered Learning in Multiagent Systems , 1997, AAAI/IAAI.

[4]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[5]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[6]  Neil Immerman,et al.  The Complexity of Decentralized Control of Markov Decision Processes , 2000, UAI.

[7]  Hector Garcia-Molina,et al.  Synchronizing a database to improve freshness , 2000, SIGMOD '00.

[8]  Martin Lauer,et al.  An Algorithm for Distributed Reinforcement Learning in Cooperative Multi-Agent Systems , 2000, ICML.

[9]  Hendrik T. Macedo,et al.  Distributed Mobile Autonomous Agents in Network Management , 2001 .

[10]  Alexis Drogoul,et al.  Multi-agent Patrolling: An Empirical Analysis of Alternative Architectures , 2002, MABS.

[11]  Kagan Tumer,et al.  Learning sequences of actions in collectives of autonomous agents , 2002, AAMAS '02.

[12]  Michail G. Lagoudakis,et al.  Coordinated Reinforcement Learning , 2002, ICML.

[13]  Martin A. Riedmiller,et al.  Using Machine Learning Techniques in Complex Multi-Agent Domains , 2003 .

[14]  A. Drogoul,et al.  Adaptive patrol for a group of robots , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[15]  Sander M. Bohte,et al.  COllective INtelligence with Sequences of Actions - Coordinating Actions in Multi-agent Systems , 2003, ECML.

[16]  Alessandro de Luna Almeida,et al.  Patrulhamento Multiagente em Grafos com Pesos , 2003 .

[17]  Craig Boutilier,et al.  Coordination in multiagent reinforcement learning: a Bayesian approach , 2003, AAMAS '03.

[18]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[19]  John D. Worth,et al.  A Modern Approach , 2005 .