Task distribution through vacancy chains for heterogeneous groups of robots

Abstract—We present an extension to our adaptive multi-robottask allocation algorithm based on vacancy chains, a resourcedistribution process common in animal and human societies. Thealgorithm uses individual reinforcement learning of task utilitiesand relies on the specializing abilities of the members of thegroup to promote dedicated optimal allocation patterns. Usingrealistic simulation experiments, we evaluate the approach bycomparing greedy and softmax action selection functions for taskallocation. We conclude that using softmax functions makes thevacancy chain algorithm responsive to different levels of abilityin a group of heterogeneous robots as well as to the effects of theunderlying group dynamics such as interference and synergy. I. I NTRODUCTION Existing multi-robot task allocation (MRTA) algorithms [1],[2], [3], [4] are typically not sensitive to the complex effectsof group dynamics, such as interference and synergy. For acooperative task such as transportation or foraging, the averagecompletion time may depend on the number of robots thatare allocated to the same task. Allocating a robot to a taskmay have either a positive or negative effect on a group’sperformance according to how much that robot contributespositively, in accomplishing tasks, or negatively, in increasinginterference. Such dynamics are difficult to model.As a way of circumventing the difficulties related to model-ing group dynamics, our past work [5] presented the vacancychain (VC) algorithm. This algorithm is inspired by the VCdistribution process as found in animal and human societies[6]. Each robot in a group following this algorithm uses localreinforcement learning (RL) to estimate the utilities of a set oftasks. From the local utilities and the robots’ action selectionfunctions, emerges the allocation pattern. Experiments in sim-ulation have shown that for groups of homogeneous robots,the VC algorithm promotes optimal system states as definedby the VC framework.The VC algorithm relies on stigmergy [7], unintentionalcommunication between the robots through their effects onthe environment, to produce specialized individuals for optimalallocation. In this paper we present experimental results show-ing how the ability of the Boltzmann Softmax action selectionfunction to differentiate between suboptimal actions can bemade to work on an inter-robot level, as a mechanism forallocating high-performance robots to high-value tasks withoutcommunication. The VC algorithm can thus be extended towork for groups of heterogeneous robots. Extending the VCalgorithm to cover groups of heterogeneous robots increasesits applicability. This is important because the VC algorithm,unlike existing MRTA algorithms, is sensitive to the effectsof group dynamics and provides a way of optimizing theperformance of groups of cooperative robots in domains wherethese effects are significant.II. M