Data-efficient Learning of Morphology and Controller for a Microrobot

Robot design is often a slow and difficult process requiring the iterative construction and testing of prototypes, with the goal of sequentially optimizing the design. For most robots, this process is further complicated by the need, when validating the capabilities of the hardware to solve the desired task, to already have an appropriate controller, which is in turn designed and tuned for the specific hardware. In this paper, we propose a novel approach, HPC-BBO, to efficiently and automatically design hardware configurations, and evaluate them by also automatically tuning the corresponding controller. HPC-BBO is based on a hierarchical Bayesian optimization process which iteratively optimizes morphology configurations (based on the performance of the previous designs during the controller learning process) and subsequently learns the corresponding controllers (exploiting the knowledge collected from optimizing for previous morphologies). Moreover, HPC-BBO can select a “batch” of multiple morphology designs at once, thus parallelizing hardware validation and reducing the number of time-consuming production cycles. We validate HPC-BBO on the design of the morphology and controller for a simulated 6-legged microrobot. Experimental results show that HPC-BBO outperforms multiple competitive baselines, and yields a 360% reduction in production cycles over standard Bayesian optimization, thus reducing the hypothetical manufacturing time of our microrobot from 21 to 4 months.

[1]  Samuel H. Brooks A Discussion of Random Methods for Seeking Maxima , 1958 .

[2]  Harold J. Kushner,et al.  A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise , 1964 .

[3]  Karl Sims,et al.  Evolving virtual creatures , 1994, SIGGRAPH.

[4]  Jordan B. Pollack,et al.  Evolutionary Body Building: Adaptive Physical Designs for Robots , 1998, Artificial Life.

[5]  Phil Husbands,et al.  Evolutionary robotics , 2014, Evolutionary Intelligence.

[6]  Jordan B. Pollack,et al.  Automatic design and manufacture of robotic lifeforms , 2000, Nature.

[7]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[8]  R. Pfeifer,et al.  Evolving Complete Agents using Artificial Ontogeny , 2003 .

[9]  Hod Lipson,et al.  Resilient Machines Through Continuous Self-Modeling , 2006, Science.

[10]  Ludovic Righetti,et al.  Design Methodologies for Central Pattern Generators: An Application to Crawling Humanoids , 2006, Robotics: Science and Systems.

[11]  Gregory S. Chirikjian,et al.  Modular Self-Reconfigurable Robot Systems [Grand Challenges of Robotics] , 2007, IEEE Robotics & Automation Magazine.

[12]  Tao Wang,et al.  Automatic Gait Optimization with Gaussian Process Regression , 2007, IJCAI.

[13]  Auke Jan Ijspeert,et al.  Central pattern generators for locomotion control in animals and robots: A review , 2008, Neural Networks.

[14]  Ronald S. Fearing,et al.  RoACH: An autonomous 2.4g crawling hexapod robot , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Auke Jan Ijspeert,et al.  Controlling swimming and crawling in a fish robot using a central pattern generator , 2008, Auton. Robots.

[16]  Zoran Popovic,et al.  Optimal gait and form for animal locomotion , 2009, ACM Trans. Graph..

[17]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[18]  E. Koonin,et al.  Is evolution Darwinian or/and Lamarckian? , 2009, Biology Direct.

[19]  Howie Choset,et al.  Using response surfaces and expected improvement to optimize snake robot gait parameters , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Andreas Krause,et al.  Parallelizing Exploration-Exploitation Tradeoffs with Gaussian Process Bandit Optimization , 2012, ICML.

[21]  Daniela Rus,et al.  M-blocks: Momentum-driven, magnetic modular robots , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Michiel van de Panne,et al.  Flexible muscle-based locomotion for bipedal creatures , 2013, ACM Trans. Graph..

[23]  Hod Lipson,et al.  Evolved Electrophysiological Soft Robots , 2014, ALIFE.

[24]  Jianwei Zhang,et al.  A Survey on CPG-Inspired Control Models and System Implementation , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Antoine Cully,et al.  Robots that can adapt like animals , 2014, Nature.

[26]  J. H. Metzen,et al.  Bayesian Optimization for Contextual Policy Search * , 2015 .

[27]  L. Theunissen,et al.  Comparative whole-body kinematics of closely related insect species with different body morphology , 2015, Journal of Experimental Biology.

[28]  Jan Peters,et al.  Bayesian optimization for learning gaits under uncertainty , 2015, Annals of Mathematics and Artificial Intelligence.

[29]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[30]  Takaya Arita,et al.  Artificial Metamorphosis: Evolutionary Design of Transforming, Soft-Bodied Robots , 2016, Artificial Life.

[31]  Hod Lipson,et al.  On the Difficulty of Co-Optimizing Morphology and Control in Evolved Virtual Creatures , 2016, ALIFE.

[32]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Tutorial , 2016, ArXiv.

[33]  Heni Ben Amor,et al.  From the Lab to the Desert: Fast Prototyping and Learning of Robot Locomotion , 2017, Robotics: Science and Systems.

[34]  Sehoon Ha,et al.  Joint Optimization of Robot Design and Motion Parameters using the Implicit Function Theorem , 2017, Robotics: Science and Systems.

[35]  A. E. Eiben,et al.  Analysis of Lamarckian evolution in morphologically evolving robots , 2017, ECAL.

[36]  Svetha Venkatesh,et al.  Process-constrained batch Bayesian optimisation , 2017, NIPS.

[37]  Kristofer S. J. Pister,et al.  First steps of a millimeter-scale walking silicon robot , 2017, 2017 19th International Conference on Solid-State Sensors, Actuators and Microsystems (TRANSDUCERS).

[38]  Sergey Levine,et al.  Learning Flexible and Reusable Locomotion Primitives for a Microrobot , 2018, IEEE Robotics and Automation Letters.

[39]  Sergey Levine,et al.  Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models , 2018, NeurIPS.

[40]  Kyrre Glette,et al.  Real-world evolution adapts robot morphology and control to hardware limitations , 2018, GECCO.

[41]  Ruben Martinez-Cantin,et al.  Funneled Bayesian Optimization for Design, Tuning and Control of Autonomous Systems , 2016, IEEE Transactions on Cybernetics.