Memetic robot control evolution and adaption to reality

Inspired by animals' ability to learn and adapt to changes in their environment during life, hybrid evolutionary algorithms have been developed and successfully applied in a number of research areas. This paper explores the effects of learning combined with a genetic algorithm to evolve control system parameters for a four-legged robot. Here, learning corresponds to the application of a local search algorithm on individuals during evolution. Two types of learning were implemented and tested, i.e. Baldwinian and Lamarckian learning. On the direct results from evolution in simulation, Lamarckian learning showed promising results, with a significant increase in final fitness compared with the results from evolution without learning. Further experiments with learning on the real robot demonstrated an efficient adaptation of the robot gait to the real world environment, and increased the performance to the level measured in simulation. This paper demonstrates that Lamarckian evolution is effective in improving the performance of robot controller evolution, and that the same learning process on the physical robot efficiently reduces the negative impact of the simulation-reality gap.

[1]  Hod Lipson,et al.  Resilient Machines Through Continuous Self-Modeling , 2006, Science.

[2]  Javier Ruiz-del-Solar,et al.  Combining Simulation and Reality in Evolutionary Robotics , 2007, J. Intell. Robotic Syst..

[3]  Hod Lipson,et al.  On the Difficulty of Co-Optimizing Morphology and Control in Evolved Virtual Creatures , 2016, ALIFE.

[4]  Stefano Nolfi,et al.  Learning to Adapt to Changing Environments in Evolving Neural Networks , 1996, Adapt. Behav..

[5]  B. Dias,et al.  Parental olfactory experience influences behavior and neural structure in subsequent generations , 2013, Nature Neuroscience.

[6]  Marco Mirolli,et al.  Evolution and Learning in an Intrinsically Motivated Reinforcement Learning Robot , 2007, ECAL.

[7]  Nicolas Bredeche,et al.  Embedded Evolutionary Robotics: The (1+1)-Restart-Online Adaptation Algorithm , 2011 .

[8]  Jeffrey L. Elman,et al.  Learning and Evolution in Neural Networks , 1994, Adapt. Behav..

[9]  Takahiro Sasaki,et al.  Adaptation toward Changing Environments: Why Darwinian in Nature? , 1997 .

[10]  Kyrre Glette,et al.  A hox gene inspired generative approach to evolving robot morphology , 2013, GECCO '13.

[11]  Kyrre Glette,et al.  Real-World Reproduction of Evolved Robot Morphologies: Automated Categorization and Evaluation , 2015, EvoApplications.

[12]  Inman Harvey,et al.  Noise and the Reality Gap: The Use of Simulation in Evolutionary Robotics , 1995, ECAL.

[13]  Geoffrey E. Hinton,et al.  How Learning Can Guide Evolution , 1996, Complex Syst..

[14]  Ferrante Neri,et al.  Memetic Compact Differential Evolution for Cartesian Robot Control , 2010, IEEE Computational Intelligence Magazine.

[15]  Karl Sims,et al.  Evolving virtual creatures , 1994, SIGGRAPH.

[16]  Bernhard Sendhoff,et al.  Lamarckian memetic algorithms: local optimum and connectivity structure analysis , 2009, Memetic Comput..

[17]  Hans-Georg Beyer,et al.  Toward a Theory of Evolution Strategies: Some Asymptotical Results from the (1,+ )-Theory , 1993, Evolutionary Computation.

[18]  Stefano Nolfi,et al.  How to Evolve Autonomous Robots: Different Approaches in Evolutionary Robotics , 1994 .

[19]  Josh C. Bongard,et al.  Evolutionary robotics , 2013, CACM.

[20]  Francesco Mondada,et al.  Evolution of Plastic Neurocontrollers for Situated Agents , 1996 .

[21]  J. Baldwin.  A New Factor in Evolution (Continued) , 1896, The American Naturalist.

[22]  Kyrre Glette,et al.  Some distance measures for morphological diversification in generative evolutionary robotics , 2014, GECCO.

[23]  L. Darrell Whitley,et al.  Lamarckian Evolution, The Baldwin Effect and Function Optimization , 1994, PPSN.

[24]  Stefano Nolfi,et al.  Learning and Evolution , 1999, Auton. Robots.

[25]  Kyrre Glette,et al.  A Comparison of Sampling Strategies for Parameter Estimation of a Robot Simulator , 2012, SIMPAR.

[26]  K. Glette,et al.  Evolution of locomotion in a simulated quadruped robot and transferral to reality , 2011 .

[27]  J. Baldwin.  A New Factor in Evolution , 1896, The American Naturalist.

[28]  Andy J. Keane,et al.  Meta-Lamarckian learning in memetic algorithms , 2004, IEEE Transactions on Evolutionary Computation.

[29]  Kyrre Glette,et al.  Filling the reality gap: Using obstacles to promote robust gaits in evolutionary robotics , 2014, 2014 IEEE International Conference on Evolvable Systems.

[30]  Hod Lipson,et al.  Evolving Dynamic Gaits on a Physical Robot , 2004 .

[31]  Stéphane Doncieux,et al.  The Transferability Approach: Crossing the Reality Gap in Evolutionary Robotics , 2013, IEEE Transactions on Evolutionary Computation.

[32]  Antoine Cully,et al.  Abstract of: "Fast Damage Recovery in Robotics with the T-Resilience Algorithm" , 2018, ALIFE.

[33]  David H. Ackley,et al.  Interactions between learning and evolution , 1991 .

[34]  Natalio Krasnogor,et al.  Memetic Algorithms , 2012, Handbook of Natural Computing.

[35]  A. E. Eiben,et al.  Evolutionary Robotics: What, Why, and Where to , 2015, Front. Robot. AI.