A Generative Hyper-Heuristic based on Multi-Objective Reinforcement Learning: the UAV Swarm Use Case

The interest in Unmanned Aerial Vehicles (UAVs) for civilian applications has seen a drastic increase in the past few years. Indeed, UAVs feature unique properties such as three-dimensional mobility and payload flexibility which provide unprecedented advantages when conducting missions like infrastructure inspection or search and rescue. However their current usage is mainly limited to a single operated or autonomous device which brings several limitations like its range of action and resilience. Using several UAVs as a swarm is one promising approach to address those limitations. However, manually designing globally efficient swarming approaches that solely rely on distributed behaviours is a complex task. The goal of this work is thus to automate the design of UAV swarming behaviours to tackle an area coverage problem. The first contribution of this work consists in modelling this problem as a multi-objective optimisation problem. The second contribution is a hyper-heuristic based on multi-objective reinforcement learning for generating distributed heuristics for that problem. Experimental results demonstrate the good stability of the generated heuristic on instances with different sizes and its capacity to well balance the multiple objectives of the optimisation problem.

[1]  Pascal Bouvry,et al.  Automating the Design of Efficient Distributed Behaviours for a Swarm of UAVs , 2020, 2020 IEEE Symposium Series on Computational Intelligence (SSCI).

[2]  Sabine Hauert,et al.  Testing the limits of pheromone stigmergy in high-density robot swarms , 2019, Royal Society Open Science.

[3]  Kevin Carey,et al.  What is A Robot Swarm: A Definition for Swarming Robotics , 2019, 2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON).

[4]  Thomas Stützle,et al.  Automatic Off-Line Design of Robot Swarms: A Manifesto , 2019, Front. Robot. AI.

[5]  Shuang Yu,et al.  A Study on Online Hyper-heuristic Learning for Swarm Robots , 2019, 2019 IEEE Congress on Evolutionary Computation (CEC).

[6]  Edmund K. Burke,et al.  A Classification of Hyper-Heuristic Approaches: Revisited , 2018, Handbook of Metaheuristics.

[7]  Serge Chaumette,et al.  Chaos-enhanced mobility models for multilevel swarms of UAVs , 2018, Swarm Evol. Comput..

[8]  Shuang Yu,et al.  Hyper-heuristic Online Learning for Self-assembling Swarm Robots , 2018, ICCS.

[9]  Pascal Bouvry,et al.  Target Tracking Optimization of UAV Swarms Based on Dual-Pheromone Clustering , 2017, 2017 3rd IEEE International Conference on Cybernetics (CYBCON).

[10]  Katia P. Sycara,et al.  Automated sequencing of swarm behaviors for supervisory control of robotic swarms , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[11]  J. Woodward,et al.  Hyper-Heuristics , 2015, GECCO.

[12]  Mauro Birattari,et al.  AutoMoDe-Chocolate: automatic design of control software for robot swarms , 2015, Swarm Intelligence.

[13]  Mauro Birattari,et al.  AutoMoDe: A novel approach to the automatic design of control software for robot swarms , 2014, Swarm Intelligence.

[14]  Michel Gendreau,et al.  Hyper-heuristics: a survey of the state of the art , 2013, J. Oper. Res. Soc..

[15]  Ann Nowé,et al.  Scalarized multi-objective reinforcement learning: Novel design techniques , 2013, 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[16]  Eliseo Ferrante,et al.  Swarm robotics: a review from the swarm engineering perspective , 2013, Swarm Intelligence.

[17]  Luca Maria Gambardella,et al.  Communication assisted navigation in robotic swarms: Self-organization and cooperation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Marco Dorigo,et al.  Path formation in a robot swarm , 2008, Swarm Intelligence.

[19]  Simin Nadjm-Tehrani,et al.  Mobility Models for UAV Group Reconnaissance Applications , 2006, 2006 International Conference on Wireless and Mobile Communications (ICWMC'06).

[20]  Graham Kendall,et al.  A Hyperheuristic Approach to Scheduling a Sales Summit , 2000, PATAT.