Guided Learning of Control Graphs for Physics-Based Characters

The difficulty of developing control strategies has been a primary bottleneck in the adoption of physics-based simulations of human motion. We present a method for learning robust feedback strategies around given motion capture clips as well as the transition paths between clips. The output is a control graph that supports real-time physics-based simulation of multiple characters, each capable of a diverse range of robust movement skills, such as walking, running, sharp turns, cartwheels, spin-kicks, and flips. The control fragments that compose the control graph are developed using guided learning. This leverages the results of open-loop sampling-based reconstruction in order to produce state-action pairs that are then transformed into a linear feedback policy for each control fragment using linear regression. Our synthesis framework allows for the development of robust controllers with a minimal amount of prior knowledge.

[1]  Jessica K. Hodgins,et al.  Animation of dynamic legged locomotion , 1991, SIGGRAPH.

[2]  David C. Brogan,et al.  Animating human athletics , 1995, SIGGRAPH.

[3]  Zoran Popovic,et al.  Physically based motion transformation , 1999, SIGGRAPH.

[4]  Petros Faloutsos,et al.  Composable controllers for physics-based character animation , 2001, SIGGRAPH.

[5]  Lucas Kovar,et al.  Motion graphs , 2002, SIGGRAPH '08.

[6]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[7]  Jehee Lee,et al.  Precomputing avatar behavior from human motion data , 2004, SCA '04.

[8]  Victor B. Zordan,et al.  Dynamic response for motion capture animation , 2005, SIGGRAPH 2005.

[9]  Victor B. Zordan,et al.  Dynamic response for motion capture animation , 2005, SIGGRAPH '05.

[10]  Jovan Popovic,et al.  Adaptation of performed ballistic motion , 2005, TOGS.

[11]  Z. Popovic,et al.  Near-optimal character animation with continuous control , 2007, ACM Trans. Graph..

[12]  Adrien Treuille,et al.  Near-optimal character animation with continuous control , 2007, SIGGRAPH 2007.

[13]  Stefan Schaal,et al.  Reinforcement learning by reward-weighted regression for operational space control , 2007, ICML '07.

[14]  KangKang Yin,et al.  SIMBICON: simple biped locomotion control , 2007, ACM Trans. Graph..

[15]  Jehee Lee,et al.  Simulating biped behaviors from human motion data , 2007, SIGGRAPH 2007.

[16]  Kwang Won Sok,et al.  Simulating biped behaviors from human motion data , 2007, ACM Trans. Graph..

[17]  M. V. D. Panne,et al.  SIMBICON: simple biped locomotion control , 2007, SIGGRAPH 2007.

[18]  Philippe Beaudoin,et al.  Continuation methods for adapting simulated skills , 2008, ACM Trans. Graph..

[19]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[20]  Jovan Popovic,et al.  Simulation of Human Motion Data using Short‐Horizon Model‐Predictive Control , 2008, Comput. Graph. Forum.

[21]  Stefan Schaal,et al.  2008 Special Issue: Reinforcement learning of motor skills with policy gradients , 2008 .

[22]  Zoran Popovic,et al.  Optimal gait and form for animal locomotion , 2009, ACM Trans. Graph..

[23]  Philippe Beaudoin,et al.  Robust task-based control policies for physics-based characters , 2009, ACM Trans. Graph..

[24]  David J. Fleet,et al.  Optimizing walking controllers , 2009, ACM Trans. Graph..

[25]  Frédo Durand,et al.  Linear Bellman combination for control of character animation , 2009, SIGGRAPH 2009.

[26]  Victor B. Zordan,et al.  Momentum control for balance , 2009, ACM Trans. Graph..

[27]  Frédo Durand,et al.  Linear Bellman combination for control of character animation , 2009, ACM Trans. Graph..

[28]  Masashi Sugiyama,et al.  Efficient Sample Reuse in EM-Based Policy Search , 2009, ECML/PKDD.

[29]  Zoran Popovic,et al.  Contact-aware nonlinear control of dynamic characters , 2009, ACM Trans. Graph..

[30]  David J. Fleet,et al.  Optimizing walking controllers for uncertain inputs and environments , 2010, ACM Trans. Graph..

[31]  M. van de Panne,et al.  Generalized biped walking control , 2010, ACM Trans. Graph..

[32]  Yoonsang Lee,et al.  Data-driven biped control , 2010, ACM Trans. Graph..

[33]  Taesoo Kwon,et al.  Control systems for human running using an inverted pendulum model and a reference motion capture sequence , 2010, SCA '10.

[34]  Martin de Lasa,et al.  Feature-based locomotion controllers , 2010, ACM Trans. Graph..

[35]  M. V. D. Panne,et al.  Sampling-based contact-rich motion control , 2010, ACM Trans. Graph..

[36]  Aaron Hertzmann,et al.  Feature-based locomotion controllers , 2010, SIGGRAPH 2010.

[37]  Martin de Lasa,et al.  Robust physics-based locomotion using low-dimensional planning , 2010, ACM Trans. Graph..

[38]  C. K. Liu,et al.  Optimal feedback control for character animation using an abstract model , 2010, ACM Trans. Graph..

[39]  C. Karen Liu,et al.  Stable Proportional-Derivative Controllers , 2011, IEEE Computer Graphics and Applications.

[40]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[41]  Zoran Popovic,et al.  Composite control of physically simulated characters , 2011, TOGS.

[42]  Sehoon Ha,et al.  Falling and landing motion control for character animation , 2012, ACM Trans. Graph..

[43]  Yuval Tassa,et al.  Synthesis and stabilization of complex behaviors through online trajectory optimization , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[44]  Zoran Popovic,et al.  Discovery of complex behaviors through contact-invariant optimization , 2012, ACM Trans. Graph..

[45]  Nicolas Pronost,et al.  Interactive Character Animation Using Simulated Physics: A State‐of‐the‐Art Review , 2012, Comput. Graph. Forum.

[46]  Vladlen Koltun,et al.  Optimizing locomotion controllers using biologically-based actuators and objectives , 2012, ACM Trans. Graph..

[47]  Sergey Levine,et al.  Guided Policy Search , 2013, ICML.

[48]  Aaron Hertzmann,et al.  Trajectory Optimization for Full-Body Movements with Complex Contacts , 2013, IEEE Transactions on Visualization and Computer Graphics.

[49]  Michiel van de Panne,et al.  Flexible muscle-based locomotion for bipedal creatures , 2013, ACM Trans. Graph..

[50]  Baining Guo,et al.  Simulation and control of skeleton-driven soft body characters , 2013, ACM Trans. Graph..

[51]  Sergey Levine,et al.  Learning Complex Neural Network Policies with Trajectory Optimization , 2014, ICML.

[52]  Emanuel Todorov,et al.  Combining the benefits of function approximation and trajectory optimization , 2014, Robotics: Science and Systems.

[53]  Eugene Fiume,et al.  Feedback control for rotational movements in feature space , 2014, Comput. Graph. Forum.

[54]  C. Karen Liu,et al.  Learning bicycle stunts , 2014, ACM Trans. Graph..

[55]  Zoran Popovic,et al.  Motion fields for interactive character locomotion , 2010, CACM.

[56]  Victor B. Zordan,et al.  Control of Rotational Dynamics for Ground and Aerial Behavior , 2014, IEEE Transactions on Visualization and Computer Graphics.

[57]  Libin Liu,et al.  Learning reduced-order feedback policies for motion skills , 2015, Symposium on Computer Animation.

[58]  Glen Berseth,et al.  Dynamic terrain traversal skills using reinforcement learning , 2015, ACM Trans. Graph..

[59]  Baining Guo,et al.  Improving Sampling‐based Motion Control , 2015, Comput. Graph. Forum.

[60]  Glen Berseth,et al.  DeepLoco , 2017, ACM Trans. Graph..