Multi-Objective Graph Heuristic Search for Terrestrial Robot Design

We present methods for co-designing rigid robots over control and morphology (including discrete topology) over multiple objectives. Previous work has addressed problems in single-objective robot co-design or multi-objective control. However, the joint multi-objective co-design problem is extremely important for generating capable, versatile, algorithmically designed robots. In this work, we present Multi-Objective Graph Heuristic Search, which extends a single-objective graph heuristic search from previous work to enable a highly efficient multi-objective search in a combinatorial design topology space. Core to this approach, we introduce a new universal, multiobjective heuristic function based on graph neural networks that is able to effectively leverage learned information between different task trade-offs. We demonstrate our approach on six combinations of seven terrestrial locomotion and design tasks, including one three-objective example. We compare the captured Pareto fronts across different methods and demonstrate that our multi-objective graph heuristic search quantitatively and qualitatively outperforms other techniques.

[1]  Sanja Fidler,et al.  Neural Graph Evolution: Towards Efficient Automatic Robot Design , 2019, ICLR.

[2]  Stefan Roth,et al.  Covariance Matrix Adaptation for Multi-objective Optimization , 2007, Evolutionary Computation.

[3]  Jure Leskovec,et al.  Hierarchical Graph Representation Learning with Differentiable Pooling , 2018, NeurIPS.

[4]  Wojciech Matusik,et al.  Prediction-Guided Multi-Objective Reinforcement Learning for Continuous Robot Control , 2020, ICML.

[5]  Konkoly Thege.  Multi-criteria Reinforcement Learning , 1998 .

[6]  Wojciech Matusik,et al.  Learning-In-The-Loop Optimization: End-To-End Control And Co-Design Of Soft Robots Through Learned Deep Latent Representations , 2019, NeurIPS.

[7]  Salim Fettaka,et al.  Design of shell-and-tube heat exchangers using multiobjective optimization , 2013 .

[8]  Carlos A. Coello Coello,et al.  A Study of the Parallelization of a Coevolutionary Multi-objective Evolutionary Algorithm , 2004, MICAI.

[9]  Tom Lenaerts,et al.  Dynamic Weights in Multi-Objective Deep Reinforcement Learning , 2018, ICML.

[10]  Sham M. Kakade,et al.  Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control , 2018, ICLR.

[11]  Sehoon Ha,et al.  Joint Optimization of Robot Design and Motion Parameters using the Implicit Function Theorem , 2017, Robotics: Science and Systems.

[12]  Jordan B. Pollack,et al.  TITLE : Generative Representations for the Automated Design of Modular Physical Robots , 2003 .

[13]  Hod Lipson,et al.  Unshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding , 2013, GECCO '13.

[14]  Daniela Rus,et al.  Functional co-optimization of articulated robots , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[15]  Matthew R. Walter,et al.  Jointly Learning to Construct and Control Agents using Deep Reinforcement Learning , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[16]  Tao Chen,et al.  How to Evaluate Solutions in Pareto-based Search-Based Software Engineering? A Critical Review and Methodological Guidance , 2020, ArXiv.

[17]  Sehoon Ha,et al.  Computational Design of Robotic Devices From High-Level Motion Specifications , 2018, IEEE Transactions on Robotics.

[18]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[19]  Gabriele Eichfelder,et al.  An Adaptive Scalarization Method in Multiobjective Optimization , 2008, SIAM J. Optim..

[20]  M. Ashby.  MULTI-OBJECTIVE OPTIMIZATION IN MATERIAL DESIGN AND SELECTION , 2000 .

[21]  Luca Bascetta,et al.  Policy gradient approaches for multi-objective sequential decision making , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[22]  Rui Wang,et al.  Deep Reinforcement Learning for Multiobjective Optimization , 2019, IEEE Transactions on Cybernetics.

[23]  Jiancheng Liu,et al.  ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  Nathan Brown,et al.  Multi-objective optimization methods in drug design. , 2013, Drug discovery today. Technologies.

[26]  Runzhe Yang,et al.  A Generalized Algorithm for Multi-Objective RL and Policy Adaptation , 2019 .

[27]  Wojciech Matusik,et al.  Interactive exploration of design trade-offs , 2018, ACM Trans. Graph..

[28]  Karl Sims,et al.  Evolving 3D Morphology and Behavior by Competition , 1994, Artificial Life.

[29]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[30]  Zoran Popovic,et al.  Optimal gait and form for animal locomotion , 2009, ACM Trans. Graph..

[31]  Qing Li,et al.  Multiobjective optimization for crash safety design of vehicles using stepwise regression model , 2008 .

[32]  Gary B. Lamont,et al.  Multiobjective evolutionary algorithms: classifications, analyses, and new innovations , 1999 .

[33]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[34]  Hod Lipson,et al.  Evolved Electrophysiological Soft Robots , 2014, ALIFE.

[35]  Shie Mannor,et al.  The Steering Approach for Multi-Criteria Reinforcement Learning , 2001, NIPS.