Advantages of Task-Specific Multi-Objective Optimisation in Evolutionary Robotics

The application of multi-objective optimisation to evolutionary robotics is receiving increasing attention. A survey of the literature reveals the different possibilities it offers to improve the automatic design of efficient and adaptive robotic systems, and points to the successful demonstrations available for both task-specific and task-agnostic approaches (i.e., with or without reference to the specific design problem to be tackled). However, the advantages of multi-objective approaches over single-objective ones have not been clearly spelled out and experimentally demonstrated. This paper fills this gap for task-specific approaches: starting from well-known results in multi-objective optimisation, we discuss how to tackle commonly recognised problems in evolutionary robotics. In particular, we show that multi-objective optimisation (i) allows evolving a more varied set of behaviours by exploring multiple trade-offs of the objectives to optimise, (ii) supports the evolution of the desired behaviour through the introduction of objectives as proxies, (iii) avoids the premature convergence to local optima possibly introduced by multi-component fitness functions, and (iv) solves the bootstrap problem exploiting ancillary objectives to guide evolution in the early phases. We present an experimental demonstration of these benefits in three different case studies: maze navigation in a single robot domain, flocking in a swarm robotics context, and a strictly collaborative task in collective robotics.

[1]  Stéphane Doncieux,et al.  New Horizons in Evolutionary Robotics: Extended Contributions from the 2009 EvoDeRob Workshop , 2011 .

[2]  Stéphane Doncieux,et al.  Beyond black-box optimization: a review of selective pressures for evolutionary robotics , 2014, Evol. Intell..

[3]  Hisao Ishibuchi,et al.  Evolutionary many-objective optimization: A short review , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[4]  Thomas Stützle,et al.  Exploratory Analysis of Stochastic Local Search Algorithms in Biobjective Optimization , 2010, Experimental Methods for the Analysis of Optimization Algorithms.

[5]  Stephane Doncieux Transfer learning for direct policy search: A reward shaping approach , 2013, 2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[6]  Stéphane Doncieux,et al.  Incremental Evolution of Animats' Behaviors as a Multi-objective Optimization , 2008, SAB.

[7]  J. Dennis,et al.  A closer look at drawbacks of minimizing weighted sums of objectives for Pareto set generation in multicriteria optimization problems , 1997 .

[8]  Francesco Mondada,et al.  The marXbot, a miniature mobile robot opening new perspectives for the collective-robotic research , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  W ReynoldsCraig Flocks, herds and schools: A distributed behavioral model , 1987 .

[10]  Gregory J. Barlow,et al.  Article in Press Robotics and Autonomous Systems ( ) – Robotics and Autonomous Systems Fitness Functions in Evolutionary Robotics: a Survey and Analysis , 2022 .

[11]  Kenneth O. Stanley,et al.  Evolving a Diversity of Creatures through Novelty Search and Local Competition , 2011 .

[12]  A. E. Eiben,et al.  Evolutionary Robotics: What, Why, and Where to , 2015, Front. Robot. AI.

[13]  Jasbir S. Arora,et al.  Survey of multi-objective optimization methods for engineering , 2004 .

[14]  Francesco Mondada,et al.  Automatic creation of an autonomous agent: genetic evolution of a neural-network driven robot , 1994 .

[15]  Genci Capi,et al.  Multiobjective Evolution of Neural Controllers and Task Complexity , 2007, IEEE Transactions on Robotics.

[16]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[17]  Hideyuki Takagi,et al.  Interactive evolutionary computation: fusion of the capabilities of EC optimization and human evaluation , 2001, Proc. IEEE.

[18]  Stéphane Doncieux,et al.  Encouraging Behavioral Diversity in Evolutionary Robotics: An Empirical Study , 2012, Evolutionary Computation.

[19]  R. Pfeifer,et al.  Self-Organization, Embodiment, and Biologically Inspired Robotics , 2007, Science.

[20]  Stéphane Doncieux,et al.  Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity , 2009, 2009 IEEE Congress on Evolutionary Computation.

[21]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[22]  Wilfried Elmenreich,et al.  On the effects of the robot configuration on evolving coordinated motion behaviors , 2013, 2013 IEEE Congress on Evolutionary Computation.

[23]  R. K. Ursem Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[24]  Eliseo Ferrante,et al.  ARGoS: a modular, parallel, multi-engine simulator for multi-robot systems , 2012, Swarm Intelligence.

[25]  Kenneth O. Stanley,et al.  Evolving a diversity of virtual creatures through novelty search and local competition , 2011, GECCO '11.

[26]  Nicola Beume,et al.  SMS-EMOA: Multiobjective selection based on dominated hypervolume , 2007, Eur. J. Oper. Res..

[27]  Antoine Cully,et al.  Abstract of: "Fast Damage Recovery in Robotics with the T-Resilience Algorithm" , 2018, ALIFE.

[28]  Gregory S. Hornby,et al.  Improving Robot Behavior Optimization by Combining User Preferences , 2014 .

[29]  Frédéric Gruau,et al.  Cellular Encoding for interactive evolutionary robotics , 1996 .

[30]  Josh C. Bongard Innocent Until Proven Guilty: Reducing Robot Shaping From Polynomial to Linear Time , 2011, IEEE Transactions on Evolutionary Computation.

[31]  Onofrio Gigliotta,et al.  Human breeders for evolving robots , 2008, Artificial Life and Robotics.

[32]  Stefano Nolfi,et al.  Engineering the Evolution of Self-Organizing Behaviors in Swarm Robotics: A Case Study , 2011, Artificial Life.

[33]  Julian Togelius,et al.  Multiobjective techniques for the use of state in genetic programming applied to simulated car racing , 2007, 2007 IEEE Congress on Evolutionary Computation.

[34]  Jean-Baptiste Mouret Novelty-Based Multiobjectivization , 2011 .

[35]  Joshua Evan Auerbach,et al.  Environmental Influence on the Evolution of Morphological Complexity in Machines , 2014, PLoS Comput. Biol..

[36]  Risto Miikkulainen,et al.  Effective diversity maintenance in deceptive domains , 2013, GECCO '13.

[37]  Hussein A. Abbass,et al.  Multiobjectivity and complexity in embodied cognition , 2005, IEEE Transactions on Evolutionary Computation.

[38]  Amiram Moshaiov,et al.  Multi-objective evolution of robot neuro-controllers , 2009, 2009 IEEE Congress on Evolutionary Computation.

[39]  Joshua D. Knowles,et al.  Modes of Problem Solving with Multiple Objectives: Implications for Interpreting the Pareto Set and for Decision Making , 2008, Multiobjective Problem Solving from Nature.

[40]  Amiram Moshaiov,et al.  Is MO-CMA-ES superior to NSGA-II for the evolution of multi-objective neuro-controllers? , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[41]  Jean-Arcady Meyer,et al.  Incremental Evolution of Target-Following Neuro-controllers for Flapping-Wing Animats , 2006, SAB.

[42]  Lothar Thiele,et al.  Multiobjective Optimization Using Evolutionary Algorithms - A Comparative Case Study , 1998, PPSN.

[43]  L. Altenberg,et al.  PERSPECTIVE: COMPLEX ADAPTATIONS AND THE EVOLUTION OF EVOLVABILITY , 1996, Evolution; international journal of organic evolution.

[44]  D. Floreano,et al.  Evolution of Adaptive Behaviour in Robots by Means of Darwinian Selection , 2010, PLoS biology.

[45]  Kalyanmoy Deb,et al.  Multiobjective optimization , 1997 .

[46]  Eckart Zitzler,et al.  HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization , 2011, Evolutionary Computation.

[47]  Katya Rodríguez-Vázquez,et al.  Autonomous robot navigation based on the evolutionary multi-objective optimization of potential fields , 2013 .

[48]  Yoseph Bar-Cohen,et al.  Biomimetics : Biologically Inspired Technologies , 2011 .

[49]  Dario Floreano,et al.  Evolutionary robots with on-line self-organization and behavioral fitness , 2000, Neural Networks.

[50]  Jürgen Branke,et al.  Interactive Multiobjective Evolutionary Algorithms , 2008, Multiobjective Optimization.

[51]  Joel Lehman,et al.  Encouraging reactivity to create robust machines , 2013, Adapt. Behav..

[52]  D. Cliff From animals to animats 3 : proceedings of the Third International Conference on Simulation of Adaptive Behavior , 1994 .

[53]  Jean-Baptiste Mouret,et al.  Stochastic optimization of a chain sliding mode controller for the mobile robot maneuvering , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[54]  Kalyanmoy Deb,et al.  Multiobjective Problem Solving from Nature: From Concepts to Applications , 2008, Natural Computing Series.

[55]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[56]  Thomas Bartz-Beielstein,et al.  Experimental Methods for the Analysis of Optimization Algorithms , 2010 .

[57]  Jeffrey L. Krichmar,et al.  Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines , 2001, Complex..

[58]  E de Margerie,et al.  Artificial evolution of the morphology and kinematics in a flapping-wing mini-UAV , 2007, Bioinspiration & biomimetics.

[59]  Lincoln Smith,et al.  Evolving controllers for a homogeneous system of physical robots: structured cooperation with minimal sensors , 2003, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[60]  Stewart W. Wilson,et al.  From Animals to Animats 5. Proceedings of the Fifth International Conference on Simulation of Adaptive Behavior , 1997 .

[61]  John Hallam,et al.  From Animals to Animats 10 , 2008 .

[62]  María Cristina Riff,et al.  Towards an immune system that solves CSP , 2007, 2007 IEEE Congress on Evolutionary Computation.

[63]  Dimitrios Papageorgiou,et al.  Robot Control for Task Performance and Enhanced Safety under Impact , 2015, Front. Robot. AI.

[64]  Stéphane Doncieux,et al.  The Transferability Approach: Crossing the Reality Gap in Evolutionary Robotics , 2013, IEEE Transactions on Evolutionary Computation.

[65]  G.J. Barlow,et al.  Incremental evolution of autonomous controllers for unmanned aerial vehicles using multi-objective genetic programming , 2004, IEEE Conference on Cybernetics and Intelligent Systems, 2004..

[66]  Dario Floreano,et al.  From Animals to Animats 2: Proceedings of the Second International Conference on Simulation of Adaptive Behavior , 2000, Journal of Cognitive Neuroscience.

[67]  Manuel López-Ibáñez,et al.  Correction: Advantages of Task-Specific Multi-Objective Optimisation in Evolutionary Robotics , 2015, PloS one.

[68]  Josh C. Bongard,et al.  The Utility of Evolving Simulated Robot Morphology Increases with Task Complexity for Object Manipulation , 2010, Artificial Life.

[69]  Carlos M. Fonseca,et al.  Inferential Performance Assessment of Stochastic Optimisers and the Attainment Function , 2001, EMO.

[70]  Jean-Baptiste Mouret,et al.  Stochastic optimization of a neural network-based controller for aggressive maneuvers on loose surfaces , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[71]  Bruce Cumings,et al.  A short review , 1983 .

[72]  Kenneth O. Stanley,et al.  Abandoning Objectives: Evolution Through the Search for Novelty Alone , 2011, Evolutionary Computation.

[73]  Stefano Nolfi,et al.  Evolving Mobile Robots Able to Display Collective Behaviors , 2003, Artificial Life.

[74]  A. Czirók,et al.  Collective Motion , 1999, physics/9902023.

[75]  Hussein A. Abbass,et al.  Automatic Generation of Controllers for Embodied Legged Organisms: A Pareto Evolutionary Multi-Objective Approach , 2004, Evolutionary Computation.

[76]  Jean-Baptiste Mouret,et al.  Optimization of humanoid walking controller: Crossing the reality gap , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[77]  Carlos M. Fonseca,et al.  The Attainment-Function Approach to Stochastic Multiobjective Optimizer Assessment and Comparison , 2010, Experimental Methods for the Analysis of Optimization Algorithms.

[78]  Vítor Matos,et al.  Multi-objective parameter CPG optimization for gait generation of a biped robot , 2013, 2013 IEEE International Conference on Robotics and Automation.

[79]  Stéphane Doncieux,et al.  With a little help from selection pressures: evolution of memory in robot controllers , 2012, ALIFE.

[80]  Stéphane Doncieux,et al.  How to promote generalisation in evolutionary robotics: the ProGAb approach , 2011, GECCO '11.

[81]  Luca Maria Gambardella,et al.  Collaboration Through the Exploitation of Local Interactions in Autonomous Collective Robotics: The Stick Pulling Experiment , 2001, Auton. Robots.

[82]  Josh C. Bongard,et al.  Combining fitness-based search and user modeling in evolutionary robotics , 2013, GECCO '13.