Multi-armed Bandit Formulation of the Task Partitioning Problem in Swarm Robotics

Task partitioning is a way of organizing work consisting in the decomposition of a task into smaller sub-tasks that can be tackled separately. Task partitioning can be beneficial in terms of reduction of physical interference, increase of efficiency, higher parallelism, and exploitation of specialization. However, task partitioning also entails costs in terms of coordination efforts and overheads that can reduce its benefits. It is therefore important to decide when to make use of task partitioning. In this paper we show that such a decision can be formulated as a multi-armed bandit problem. This is advantageous since the theoretical properties of the multi-armed bandit problem are well understood and several algorithms have been proposed for tackling it. We carry out our study in simulation, using a swarm robotics foraging scenario as a testbed. We test an ad-hoc algorithm and two algorithms proposed in the literature for multi-armed bandit problems. The results confirm that the problem of selecting whether to partition a task can be formulated as a multi-armed bandit problem and tackled with existing algorithms.

[1]  Richard T. Vaughan,et al.  Adaptive multi-robot bucket brigade foraging , 2008, ALIFE.

[2]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[3]  R. Agrawal.  Sample mean based index policies by O(log n) regret for the multi-armed bandit problem , 1995, Advances in Applied Probability.

[4]  Eliseo Ferrante,et al.  Swarmanoid: A Novel Concept for the Study of Heterogeneous Robotic Swarms , 2013, IEEE Robotics & Automation Magazine.

[5]  Marco Dorigo,et al.  Swarm Robotics: Special Issue Editorial , 2004 .

[6]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[7]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[8]  F. Ratnieks,et al.  Task partitioning in insect societies , 1999, Insectes Sociaux.

[9]  Richard T. Vaughan,et al.  Adapting to non-uniform resource distributions in robotic swarm foraging through work-site relocation , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Mauro Birattari,et al.  Task Partitioning in Swarms of Robots: Reducing Performance Losses Due to Interference at Shared Resources , 2011 .

[11]  Maja J. Mataric,et al.  On foraging strategies for large-scale multi-robot systems , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Mauro Birattari,et al.  Self-organized Task Partitioning in a Swarm of Robots , 2010, ANTS Conference.

[13]  Alessandro Giua,et al.  Guest Editorial , 2001, Discrete event dynamic systems.

[14]  P. W. Jones,et al.  Bandit Problems, Sequential Allocation of Experiments , 1987 .

[15]  Mauro Birattari,et al.  The IRIDIA TAM: A device for task abstraction for the e-puck robot , 2010 .

[16]  R. Jeanne.  THE EVOLUTION OF THE ORGANIZATION OF WORK IN SOCIAL INSECTS , 2013 .

[17]  Eliseo Ferrante,et al.  ARGoS: A modular, multi-engine simulator for heterogeneous swarm robotics , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Pattie Maes,et al.  A Study of Territoriality: The Role of Critical Mass in Adaptive Task Division , 1996 .

[19]  Mauro Birattari,et al.  Task partitioning in swarms of robots: an adaptive method for strategy selection , 2011, Swarm Intelligence.

[20]  Gaurav S. Sukhatme,et al.  Emergent bucket brigading: a simple mechanisms for improving performance in multi-robot constrained-space foraging tasks , 2001, AGENTS '01.