Neural Network Dynamics Models for Control of Under-actuated Legged Millirobots

Millirobots are a promising robotic platform for many applications due to their small size and low manufacturing costs. However, controlling these millirobots is difficult due to their underactuation, power constraints, and size. While hand-engineered controllers can sometimes control these millirobots, they often have difficulties with highly dynamic maneuvers and complex terrains. We present a learning based approach in which a model of the dynamics is learned from data gathered by the millirobot, and that data is then leveraged by an MPC controller. We show that with 17 minutes of random data collected with the VelociRoACH millirobot, the VelociRoACH can accurately follow trajectories at higher speeds and on more difficult terrains than a differential drive controller. Experiment videos can be found at this https URL

[1]  Ronald S. Fearing,et al.  RoACH: An autonomous 2.4g crawling hexapod robot , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Ronald S. Fearing,et al.  Sliding and hopping gaits for the underactuated Acrobot , 1998, IEEE Trans. Robotics Autom..

[3]  Stefan Schaal,et al.  Fast, robust quadruped locomotion over challenging terrain , 2010, 2010 IEEE International Conference on Robotics and Automation.

[4]  Howie Choset,et al.  Using response surfaces and expected improvement to optimize snake robot gait parameters , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Katie Byl,et al.  Metastable legged-robot locomotion , 2008 .

[6]  Daniel E. Koditschek,et al.  A Physical Model for Dynamical Arthropod Running on Level Ground , 2008, ISER.

[7]  Duncan W. Haldane,et al.  Automatic identification of dynamic piecewise affine models for a running robot , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Florentin Wörgötter,et al.  Neuromechanical control for hexapedal robot walking on challenging surfaces and surface classification , 2014, Robotics Auton. Syst..

[9]  Ronald S. Fearing,et al.  Fast scale prototyping for folded millirobots , 2008, ICRA.

[10]  Peter Fankhauser,et al.  ANYmal - a highly mobile and dynamic quadrupedal robot , 2016, IROS 2016.

[11]  Duncan W. Haldane,et al.  Running beyond the bio-inspired regime , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Daniel E. Koditschek,et al.  Characterization of Dynamic Behaviors in a Hexapod Robot , 2010, ISER.

[13]  Honglak Lee,et al.  Control of Memory, Active Perception, and Action in Minecraft , 2016, ICML.

[14]  Auke Jan Ijspeert,et al.  Learning robot gait stability using neural networks as sensory feedback function for Central Pattern Generators , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Jan Peters,et al.  Toward fast policy search for learning legged locomotion , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[17]  Kikuo Fujimura,et al.  The intelligent ASIMO: system overview and integration , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Tao Wang,et al.  Automatic Gait Optimization with Gaussian Process Regression , 2007, IJCAI.

[19]  Duncan W. Haldane,et al.  Animal-inspired design and aerodynamic stabilization of a hexapedal millirobot , 2013, 2013 IEEE International Conference on Robotics and Automation.

[20]  Sergey Levine,et al.  Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[21]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[22]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[23]  Sergey Levine,et al.  Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Anil V. Rao,et al.  ( Preprint ) AAS 09-334 A SURVEY OF NUMERICAL METHODS FOR OPTIMAL CONTROL , 2009 .

[25]  Michael L. Littman,et al.  Perception-based generalization in model-based reinforcement learning , 2009 .

[26]  Jan Peters,et al.  An experimental comparison of Bayesian optimization for bipedal locomotion , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[27]  John Guckenheimer,et al.  The Dynamics of Legged Locomotion: Models, Analyses, and Challenges , 2006, SIAM Rev..

[28]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[29]  Ruben Grandia,et al.  Contact Invariant Model Learning for Legged Robot Locomotion , 2018, IEEE Robotics and Automation Letters.

[30]  Dieter Fox,et al.  GP-BayesFilters: Bayesian filtering using Gaussian process prediction and observation models , 2008, IROS.

[31]  Jonathan E. Clark,et al.  iSprawl: Design and Tuning for High-speed Autonomous Open-loop Running , 2006, Int. J. Robotics Res..

[32]  H. Benjamin Brown,et al.  c ○ 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. RHex: A Biologically Inspired Hexapod Runner ∗ , 2022 .

[33]  Thomas Kindermann,et al.  Walknet--a biologically inspired network to control six-legged walking , 1998, Neural Networks.

[34]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[35]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[36]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[37]  Philip Holmes,et al.  Stability Analysis of a Clock-Driven Rigid-Body SLIP Model for RHex , 2004, Int. J. Robotics Res..

[38]  Manuela M. Veloso,et al.  An evolutionary approach to gait learning for four-legged robots , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[39]  H. Sebastian Seung,et al.  Learning to Walk in 20 Minutes , 2005 .

[40]  Jonathan E. Clark,et al.  Biomimetic design and fabrication of a hexapedal running robot , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[41]  Duncan W. Haldane,et al.  Roll oscillation modulated turning in dynamic millirobots , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[42]  Kevin Blankespoor,et al.  BigDog, the Rough-Terrain Quadruped Robot , 2008 .

[43]  Michael C. Yip,et al.  Model-Less Feedback Control of Continuum Manipulators in Constrained Environments , 2014, IEEE Transactions on Robotics.

[44]  Pieter Abbeel,et al.  Hierarchical Apprenticeship Learning with Application to Quadruped Locomotion , 2007, NIPS.

[45]  Samuel Burden,et al.  Bio-inspired design and dynamic maneuverability of a minimally actuated six-legged robot , 2010, 2010 3rd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics.

[46]  Christopher G. Atkeson,et al.  Optimization and learning for rough terrain legged locomotion , 2011, Int. J. Robotics Res..

[47]  David Zarrouk,et al.  Dynamic legged locomotion for palm-size robots , 2015, Defense + Security Symposium.