Evolutionary online behaviour learning and adaptation in robotic systems

In this thesis, we study new ways to enable efficient online learning in autonomous robots. We employ a control synthesis methodology called evolutionary robotics, which emerged in the 1990s as a promising alternative to classic artificial intelligence techniques and design methodologies for control systems. In online learning through evolution, henceforth called online evolution, an evolutionary algorithm is executed onboard each robot in order to create and continuously optimise its behavioural control logic. Each instance of the evolutionary algorithm executes without any external supervision or human intervention. Online evolution can thus automatically generate the artificial intelligence that controls each robot, and creates the potential for long-term behaviour adaptation and learning: robots can continuously self-adjust and learn new behaviours in response to, for example, changes in the task requirements or environmental conditions, and to faults in the sensors and/or actuators. Despite the potential for automatic behaviour learning, online evolution is not frequently employed for a number of reasons. First, online evolution typically requires several hours or days to synthesise solutions to a task. As a result, the approach has not yet been practically exploited in real-robot systems. Second, one common assumption in the field is that online evolution enables continuous learning and adaptation to previously unforeseen circumstances. However, only a small number of ad-hoc experiments have been carried out in simulation. That is, the potential for online evolution to enable dynamic adaptation and learning has been largely left unstudied. The main goal of this thesis is to address some of the fundamental issues associated with online evolution to bring it closer to widespread adoption. Our research focuses on studying if and how to accelerate and increase the performance of online evolution. Our first research contribution is a comprehensive presentation and analysis of Online Decentralised NeuroEvolution of Augmenting Topologies (odNEAT), an algorithm for online evolution of neural networkbased controllers in multirobot systems. odNEAT differs from more traditional approaches to online evolution because both the weighting parameters and the topological structure of neural networks are under evolutionary control. Our second research contribution focuses on investigating the dynamics of online evolution of controllers at two different levels. At the microscopic scale, we assess the dynamics of distinct neuronal models. At the macroscopic scale, we investigate the scalability properties of online evolution with respect to group size. The outcomes of the contribution are an assertion of the critical role of the controller evaluation policy, and an analysis of how the group size influences task performance. In our third research contribution, we capitalise on the knowledge gained from the second study, and we introduce: (i) a racing approach that allows individual robots to cut short the evaluation of poor controllers, (ii) a population cloning approach that enables each individual robot to clone and transmit a varying number of high-performing controllers to other robots nearby, and (iii) online hyper-evolution (OHE), an unprecedented approach in evolutionary robotics with the capability to automatically construct algorithms for controller generation during task execution. To conclude, we validate our research in real robotic hardware, and we successfully demonstrate evolution of controllers to solve three classic evolutionary robotics tasks in a timely manner (one hour or less).