Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robots

Legged locomotion is commonly studied and expressed as a discrete set of gait patterns, like walk, trot, gallop, which are usually treated as given and pre-programmed in legged robots for efficient locomotion at different speeds. However, fixing a set of pre-programmed gaits limits the generality of locomotion. Recent animal motor studies show that these conventional gaits are only prevalent in ideal flat terrain conditions while real-world locomotion is unstructured and more like bouts of intermittent steps. What principles could lead to both structured and unstructured patterns across mammals and how to synthesize them in robots? In this work, we take an analysis-by-synthesis approach and learn to move by minimizing mechanical energy. We demonstrate that learning to minimize energy consumption plays a key role in the emergence of natural locomotion gaits at different speeds in real quadruped robots. The emergent gaits are structured in ideal terrains and look similar to that of horses and sheep. The same approach leads to unstructured gaits in rough terrains which is consistent with the findings in animal motor control. We validate our hypothesis in both simulation and real hardware across natural terrains. Videos at https://energy-locomotion.github.io.

[1]  Jitendra Malik,et al.  RMA: Rapid Motor Adaptation for Legged Robots , 2021, Robotics: Science and Systems.

[2]  Byron Boots,et al.  Fast and Efficient Locomotion via Learned Gait Transitions , 2021, CoRL.

[3]  Manoj Srinivasan,et al.  Computer optimization of a minimal biped model discovers walking and running , 2006, Nature.

[4]  Zoran Popovic,et al.  Contact-invariant optimization for hand manipulation , 2012, SCA '12.

[5]  M H Raibert,et al.  Trotting, pacing and bounding by a quadruped robot. , 1990, Journal of biomechanics.

[6]  John E A Bertram,et al.  Constrained optimization in human walking: cost minimization and gait plasticity , 2005, Journal of Experimental Biology.

[7]  Byron Boots,et al.  Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion , 2020, Conference on Robot Learning.

[8]  John E A Bertram,et al.  An inelastic quadrupedal model discovers four-beat walking, two-beat running, and pseudo-elastic actuation as energetically optimal , 2019, PLoS Comput. Biol..

[9]  Chelsea Finn,et al.  Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[10]  C. Vaughan,et al.  Froude and the contribution of naval architecture to our understanding of bipedal locomotion. , 2005, Gait & posture.

[11]  David E. Orin,et al.  Intelligent control of quadruped gallops , 2003 .

[12]  Hartmut Witte,et al.  Comparing the effect of different spine and leg designs for a small bounding quadruped robot , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[13]  R. Alexander,et al.  Mechanics of locomotion of dogs (Canis familiaris) and sheep (Ovis aries). , 2009, Journal of zoology.

[14]  C. Karen Liu,et al.  Sim-to-Real Transfer for Biped Locomotion , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[15]  Sergey Levine,et al.  Learning to Walk via Deep Reinforcement Learning , 2018, Robotics: Science and Systems.

[16]  Atil Iscen,et al.  Sim-to-Real: Learning Agile Locomotion For Quadruped Robots , 2018, Robotics: Science and Systems.

[17]  Joonho Lee,et al.  Learning agile and dynamic motor skills for legged robots , 2019, Science Robotics.

[18]  Russ Tedrake,et al.  A direct method for trajectory optimization of rigid bodies through contact , 2014, Int. J. Robotics Res..

[19]  D. F. Hoyt,et al.  Gait and the energetics of locomotion in horses , 1981, Nature.

[20]  R. McN. Alexander,et al.  The Gaits of Bipedal and Quadrupedal Animals , 1984 .

[21]  Lorenz Wellhausen,et al.  Learning quadrupedal locomotion over challenging terrain , 2020, Science Robotics.

[22]  K. Adolph,et al.  Bouts of steps: The organization of infant exploration. , 2016, Developmental psychobiology.

[23]  C. Karen Liu,et al.  Learning symmetric and low-energy locomotion , 2018, ACM Trans. Graph..

[24]  R. Alexander,et al.  A dynamic similarity hypothesis for the gaits of quadrupedal mammals , 2009 .

[25]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[26]  Marco Hutter,et al.  Per-Contact Iteration Method for Solving Contact Dynamics , 2018, IEEE Robotics and Automation Letters.

[27]  Sangbae Kim,et al.  Dynamic Locomotion in the MIT Cheetah 3 Through Convex Model-Predictive Control , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[28]  Ioannis Havoutis,et al.  Real-Time Trajectory Adaptation for Quadrupedal Locomotion using Deep Reinforcement Learning , 2021, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[29]  J. MUYBRIDGE,et al.  The Horse in Motion , 1882, Nature.

[30]  Scott Kuindersma,et al.  Variational Contact-Implicit Trajectory Optimization , 2017, ISRR.

[31]  J. Bertram,et al.  Reducing gravity takes the bounce out of running , 2018, Journal of Experimental Biology.

[32]  Jesse M. Lingeman,et al.  How Do You Learn to Walk? Thousands of Steps and Dozens of Falls per Day , 2012, Psychological science.

[33]  Z. Afelt,et al.  Speed control in animal locomotion: transitions between symmetrical and nonsymmetrical gaits in the dog. , 1983, Acta neurobiologiae experimentalis.

[34]  Sergey Levine,et al.  High-Dimensional Continuous Control Using Generalized Advantage Estimation , 2015, ICLR.

[35]  M Hildebrand,et al.  Symmetrical gaits of horses. , 1965, Science.

[36]  Greg Turk,et al.  Preparing for the Unknown: Learning a Universal Policy with Online System Identification , 2017, Robotics: Science and Systems.

[37]  Jie Tan,et al.  Learning Agile Robotic Locomotion Skills by Imitating Animals , 2020, RSS 2020.