Physical Scaffolding Accelerates the Evolution of Robot Behavior

In some evolutionary robotics experiments, evolved robots are transferred from simulation to reality, while sensor/motor data flows back from reality to improve the next transferral. We envision a generalization of this approach: a simulation-to-reality pipeline. In this pipeline, increasingly embodied agents flow up through a sequence of increasingly physically realistic simulators, while data flows back down to improve the next transferral between neighboring simulators; physical reality is the last link in this chain. As a first proof of concept, we introduce a two-link chain: A fast yet low-fidelity (lo-fi) simulator hosts minimally embodied agents, which gradually evolve controllers and morphologies to colonize a slow yet high-fidelity (hi-fi) simulator. The agents are thus physically scaffolded. We show here that, given the same computational budget, these physically scaffolded robots reach higher performance in the hi-fi simulator than do robots that only evolve in the hi-fi simulator, but only for a sufficiently difficult task. These results suggest that a simulation-to-reality pipeline may strike a good balance between accelerating evolution in simulation while anchoring the results in reality, free the investigator from having to prespecify the robot's morphology, and pave the way to scalable, automated, robot-generating systems.

[1]  Pattie Maes,et al.  Toward the Evolution of Dynamical Neural Networks for Minimally Cognitive Behavior , 1996 .

[2]  Stéphane Doncieux,et al.  The Transferability Approach: Crossing the Reality Gap in Evolutionary Robotics , 2013, IEEE Transactions on Evolutionary Computation.

[3]  Di Paolo,et al.  Homeostatic adaptation to inversion of the visual field and other sensorimotor disruptions , 2000 .

[4]  Stéphane Doncieux,et al.  Crossing the reality gap in evolutionary robotics by promoting transferable controllers , 2010, GECCO '10.

[5]  Jeffrey L. Krichmar,et al.  Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines , 2001, Complex..

[6]  Inman Harvey,et al.  An Evolutionary Ecological Approach to the Study of Learning Behavior Using a Robot-Based Model , 2002, Adapt. Behav..

[7]  Xabier E. Barandiaran,et al.  A Dynamical Systems Account of Sensorimotor Contingencies , 2013, Front. Psychol..

[8]  Josh C. Bongard,et al.  Not all physics simulators can be wrong in the same way , 2012, GECCO '12.

[9]  Joel Lehman,et al.  Encouraging reactivity to create robust machines , 2013, Adapt. Behav..

[10]  Jordan B. Pollack,et al.  Evolutionary Techniques in Physical Robotics , 2000, ICES.

[11]  Thomas Bräunl,et al.  Evolving Autonomous Biped Control from Simulation to Reality , 2004 .

[12]  Masahiro Fujita,et al.  Evolving robust gaits with AIBO , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[13]  Jordan B. Pollack,et al.  The GOLEM project: evolving hardware bodies and brains , 2000, Proceedings. The Second NASA/DoD Workshop on Evolvable Hardware.

[14]  John Rieffel,et al.  EvoFab: A Fully Embodied Evolutionary Fabricator , 2010, ICES.

[15]  Mauro Birattari,et al.  AutoMoDe: A novel approach to the automatic design of control software for robot swarms , 2014, Swarm Intelligence.

[16]  Dario Floreano,et al.  Evolution of Plastic Control Networks , 2001, Auton. Robots.

[17]  F. Iida,et al.  Morphological Evolution of Physical Robots through Model-Free Phenotype Development , 2015, PloS one.

[18]  Hideyuki Ando,et al.  Extended homeostatic adaptation model with metabolic causation in plasticity mechanism—toward constructing a dynamic neural network model for mental imagery , 2013, Adapt. Behav..

[19]  Inman Harvey,et al.  Noise and the Reality Gap: The Use of Simulation in Evolutionary Robotics , 1995, ECAL.

[20]  Nick Jakobi,et al.  Minimal simulations for evolutionary robotics , 1998 .

[21]  Marco Colombetti,et al.  Robot Shaping: An Experiment in Behavior Engineering , 1997 .

[22]  Hod Lipson,et al.  Resilient Machines Through Continuous Self-Modeling , 2006, Science.

[23]  Josh Bongard,et al.  Morphological change in machines accelerates the evolution of robust behavior , 2011, Proceedings of the National Academy of Sciences.

[24]  Nick Jacobi,et al.  Running Across the Reality Gap: Octopod Locomotion Evolved in a Minimal Simulation , 1998, EvoRobot.

[25]  Randall D. Beer,et al.  Integrating reactive, sequential, and learning behavior using dynamical neural networks , 1994 .

[26]  David S. Touretzky,et al.  Shaping robot behavior using principles from instrumental conditioning , 1997, Robotics Auton. Syst..

[27]  Randall D. Beer,et al.  Evolving Dynamical Neural Networks for Adaptive Behavior , 1992, Adapt. Behav..

[28]  Stefano Nolfi,et al.  Evolving mobile robots in simulated and real environments , 1995 .

[29]  Marco Colombetti,et al.  Robot Shaping: Developing Autonomous Agents Through Learning , 1994, Artif. Intell..

[30]  Nick Jakobi,et al.  Evolutionary Robotics and the Radical Envelope-of-Noise Hypothesis , 1997, Adapt. Behav..

[31]  Eduardo Izquierdo-Torres,et al.  Analysis of a Dynamical Recurrent Neural Network Evolved for Two Qualitatively Different Tasks: Walking and Chemotaxis , 2008, ALIFE.

[32]  Chandana Paul,et al.  Morphological computation: A basis for the analysis of morphology and control requirements , 2006, Robotics Auton. Syst..

[33]  R. Pfeifer,et al.  Morphological computation for adaptive behavior and cognition , 2006 .

[34]  Satinder Singh.  Transfer of Learning by Composing Solutions of Elemental Sequential Tasks , 1992, Mach. Learn..

[35]  Hod Lipson,et al.  Age-fitness pareto optimization , 2010, GECCO '10.

[36]  Helmut Hauser,et al.  Towards a theoretical foundation for morphological computation with compliant bodies , 2011, Biological Cybernetics.

[37]  Randall D. Beer,et al.  Further Experiments in the Evolution of Minimally Cognitive Behavior: From Perceiving Affordances to Selective Attention , 2000 .

[38]  Antoine Cully,et al.  Robots that can adapt like animals , 2014, Nature.

[39]  Luc Berthouze,et al.  Motor Skill Acquisition Under Environmental Perturbations: On the Necessity of Alternate Freezing and Freeing of Degrees of Freedom , 2004, Adapt. Behav..

[40]  Stéphane Doncieux,et al.  Beyond black-box optimization: a review of selective pressures for evolutionary robotics , 2014, Evol. Intell..

[41]  A. Cangelosi,et al.  Developmental Robotics: From Babies to Robots , 2015 .

[42]  Javier Ruiz-del-Solar,et al.  Back to reality: Crossing the reality gap in evolutionary robotics , 2004 .