Motor Babble: Morphology-Driven Coordinated Control of Articulated Characters

Locomotion in humans and animals is highly coordinated, with many joints moving together. Learning similar coordinated locomotion in articulated virtual characters, in the absence of reference motion data, is a challenging task due to the high number of degrees of freedom and the redundancy that comes with it. In this paper, we present a method for learning locomotion for virtual characters in a low dimensional latent space which defines how different joints move together. We introduce a technique called motor babble, wherein a character interacts with its environment by actuating its joints through uncoordinated, low-level (motor) excitations, resulting in a corpus of motion data from which a manifold latent space is extracted. Dimensions of the extracted manifold define a wide variety of synergies pertaining to the character and, through reinforcement learning, we train the character to learn locomotion in the latent space by selecting a small set of appropriate latent dimensions, along with learning the corresponding policy.

[1]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[2]  Shigeo Hirose,et al.  Biologically Inspired Robots: Snake-Like Locomotors and Manipulators , 1993 .

[3]  Auke Jan Ijspeert,et al.  AmphiBot I: an amphibious snake-like robot , 2005, Robotics Auton. Syst..

[4]  Zoran Popovic,et al.  Optimal gait and form for animal locomotion , 2009, ACM Trans. Graph..

[5]  Peter Stone,et al.  Policy gradient reinforcement learning for fast quadrupedal locomotion , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[6]  Jungdam Won,et al.  Aerobatics control of flying creatures via self-regulated learning , 2018, ACM Trans. Graph..

[7]  Jessica K. Hodgins,et al.  Animation of dynamic legged locomotion , 1991, SIGGRAPH.

[8]  Taku Komura,et al.  Phase-functioned neural networks for character control , 2017, ACM Trans. Graph..

[9]  Glen Berseth,et al.  DeepLoco , 2017, ACM Trans. Graph..

[10]  Giulio Sandini,et al.  Active learning for multiple sensorimotor coordination based on state confidence , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Fumiya Iida,et al.  "Cheap" Rapid Locomotion of a Quadruped Robot: Self-Stabilization of Bounding Gait , 2004 .

[12]  Pei Xu,et al.  Low Dimensional Motor Skill Learning Using Coactivation , 2019, MIG.

[13]  Jungdam Won,et al.  A scalable approach to control diverse behaviors for physically simulated characters , 2020, ACM Trans. Graph..

[14]  Auke Jan Ijspeert,et al.  Central pattern generators for locomotion control in animals and robots: A review , 2008, Neural Networks.

[15]  Sergey Levine,et al.  Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor , 2018, ICML.

[16]  Greg Turk,et al.  Articulated swimming creatures , 2011, SIGGRAPH 2011.

[17]  Marc H. Raibert,et al.  Legged robots , 1986, CACM.

[18]  Sunmin Lee,et al.  Learning predict-and-simulate policies from unorganized human motion data , 2019, ACM Trans. Graph..

[19]  Glen Berseth,et al.  Terrain-adaptive locomotion skills using deep reinforcement learning , 2016, ACM Trans. Graph..

[20]  Zoran Popovic,et al.  Generalizing locomotion style to new animals with inverse optimal regression , 2014, ACM Trans. Graph..

[21]  C. Karen Liu,et al.  Learning symmetric and low-energy locomotion , 2018, ACM Trans. Graph..

[22]  Jungdam Won,et al.  How to train your dragon , 2017, ACM Trans. Graph..

[23]  CARL , 2020 .

[24]  Michiel van de Panne,et al.  Parameterized gait synthesis , 1996, IEEE Computer Graphics and Applications.

[25]  Jehee Lee,et al.  Simulating biped behaviors from human motion data , 2007, SIGGRAPH 2007.

[26]  Richard J. Beach,et al.  Proceedings of the 15th annual conference on Computer graphics and interactive techniques , 1988, International Conference on Computer Graphics and Interactive Techniques.

[27]  Auke Jan Ijspeert,et al.  AmphiBot II: An Amphibious Snake Robot that Crawls and Swims using a Central Pattern Generator , 2006 .

[28]  Zoran Popovic,et al.  Realistic modeling of bird flight animations , 2003, ACM Trans. Graph..

[29]  Nicolas Pronost,et al.  Interactive Character Animation Using Simulated Physics: A State‐of‐the‐Art Review , 2012, Comput. Graph. Forum.

[30]  Sergey Levine,et al.  DeepMimic , 2018, ACM Trans. Graph..

[31]  Joaquim B. Cavalcante Neto,et al.  Using natural vibrations to guide control for locomotion , 2012, I3D '12.

[32]  Trista Pei-chun Chen,et al.  CARL , 2020, ACM Trans. Graph..

[33]  Demetri Terzopoulos,et al.  Automated learning of muscle-actuated locomotion through control abstraction , 1995, SIGGRAPH.

[34]  Pieter Abbeel,et al.  Benchmarking Deep Reinforcement Learning for Continuous Control , 2016, ICML.

[35]  Jitendra Malik,et al.  SFV , 2018, ACM Trans. Graph..

[36]  A. Ijspeert,et al.  From Swimming to Walking with a Salamander Robot Driven by a Spinal Cord Model , 2007, Science.

[37]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[38]  Michiel van de Panne,et al.  Virtual Wind-up Toys for Animation , 1993 .

[39]  Sergey Levine,et al.  MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policies , 2019, NeurIPS.

[40]  C. Karen Liu,et al.  Modal-space control for articulated characters , 2011, TOGS.

[41]  Gavin S. P. Miller,et al.  The motion dynamics of snakes and worms , 1988, SIGGRAPH.

[42]  Marie-Paule Cani,et al.  Modal Locomotion: Animating Virtual Characters with Natural Vibrations , 2009, Comput. Graph. Forum.

[43]  K. Wampler,et al.  Optimal gait and form for animal locomotion , 2009, SIGGRAPH 2009.

[44]  Jun-yong Noh,et al.  Data-driven control of flapping flight , 2013, TOGS.

[45]  S. Levine,et al.  DeepMimic , 2018, ACM Transactions on Graphics.

[46]  Auke Jan Ijspeert,et al.  Online Optimization of Swimming and Crawling in an Amphibious Snake Robot , 2008, IEEE Transactions on Robotics.

[47]  Jungdam Won,et al.  SoftCon: simulation and control of soft-bodied animals with biomimetic actuators , 2019, ACM Trans. Graph..

[48]  Jehee Lee,et al.  Data-driven biped control , 2010, SIGGRAPH 2010.

[49]  Sergey Levine,et al.  High-Dimensional Continuous Control Using Generalized Advantage Estimation , 2015, ICLR.