Automating the Design of Efficient Distributed Behaviours for a Swarm of UAVs

The usage of Unmanned Aerial Vehicles (UAVs) is gradually gaining momentum for commercial applications. The vast majority considers a single UAV, which comes with several constraints such as its range of operations or the number of sensors it can carry. Using multiple autonomous UAVs simultaneously as a swarm makes it possible to overcome these limitations. However, manually designing complex emerging behaviours like swarming is a difficult and tedious task especially for such distributed systems which performance is hardly predictable. This article therefore proposes to automate the design of UAV swarming behaviours by defining a multi-objective optimisation problem, so called Coverage of a Connected-UAV Swarm (CCUS), and designing a Q-Learning based Hyper-Heuristic (QLHH) for generating distributed CCUS heuristics. Experimental results demonstrate the capacity of QLHH to generate efficient heuristics for any instance from a given class.

[1]  Matthias R. Brust,et al.  Dynamic multi-hop clustering for mobile hybrid wireless networks , 2008, ICUIMC '08.

[2]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[3]  Ann Nowé,et al.  Multi-objective reinforcement learning using sets of pareto dominating policies , 2014, J. Mach. Learn. Res..

[4]  Ann Nowé,et al.  Scalarized multi-objective reinforcement learning: Novel design techniques , 2013, 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[5]  Pascal Bouvry,et al.  Management of an academic HPC cluster: The UL experience , 2014, 2014 International Conference on High Performance Computing & Simulation (HPCS).

[6]  Le Song,et al.  2 Common Formulation for Greedy Algorithms on Graphs , 2018 .

[7]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[8]  Michel Gendreau,et al.  Hyper-heuristics: a survey of the state of the art , 2013, J. Oper. Res. Soc..

[9]  Pascal Bouvry,et al.  Tackling Large-Scale and Combinatorial Bi-Level Problems With a Genetic Programming Hyper-Heuristic , 2020, IEEE Transactions on Evolutionary Computation.

[10]  Nikola Miskovic,et al.  Heuristics pool for hyper-heuristic selection during task allocation in a heterogeneous swarm of marine robots , 2018 .

[11]  Andries Petrus Engelbrecht,et al.  Analysis of selection hyper-heuristics for population-based meta-heuristics in real-valued dynamic optimization , 2018, Swarm Evol. Comput..

[12]  Pascal Bouvry,et al.  UAV Fleet Mobility Model with Multiple Pheromones for Tracking Moving Observation Targets , 2016, EvoApplications.

[13]  Ramón Rizo,et al.  Modelling multi-rotor UAVs swarm deployment using virtual pheromones , 2018, PloS one.

[14]  M.,et al.  Automated Design of Metaheuristic Algorithms ? , 2018 .

[15]  Pascal Bouvry,et al.  Connectivity Stability in Autonomous Multi-level UAV Swarms for Wide Area Monitoring , 2015, DIVANet@MSWiM.

[16]  Pascal Bouvry,et al.  Target Tracking Optimization of UAV Swarms Based on Dual-Pheromone Clustering , 2017, 2017 3rd IEEE International Conference on Cybernetics (CYBCON).

[17]  Ender Özcan,et al.  A genetic programming hyper-heuristic for the multidimensional knapsack problem , 2014, Kybernetes.

[18]  Kevin Carey,et al.  What is A Robot Swarm: A Definition for Swarming Robotics , 2019, 2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON).

[19]  Pascal Bouvry,et al.  A GP Hyper-Heuristic Approach for Generating TSP Heuristics , 2019, 2019 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW).

[20]  Graham Kendall,et al.  A Hyperheuristic Approach to Scheduling a Sales Summit , 2000, PATAT.

[21]  Dewen Hu,et al.  Multiobjective Reinforcement Learning: A Comprehensive Overview , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[22]  Peter Stone,et al.  Multiagent Learning Paradigms , 2017, EUMAS/AT.

[23]  Bart De Schutter,et al.  Multi-agent Reinforcement Learning: An Overview , 2010 .

[24]  Thomas Stützle,et al.  Automatic Off-Line Design of Robot Swarms: A Manifesto , 2019, Front. Robot. AI.