Learning time-critical responses for interactive character control

Creating agile and responsive characters from a collection of unorganized human motion has been an important problem of constructing interactive virtual environments. Recently, learning-based approaches have successfully been exploited to learn deep network policies for the control of interactive characters. The agility and responsiveness of deep network policies are influenced by many factors, such as the composition of training datasets, the architecture of network models, and learning algorithms that involve many threshold values, weights, and hyper-parameters. In this paper, we present a novel teacher-student framework to learn time-critically responsive policies, which guarantee the time-to-completion between user inputs and their associated responses regardless of the size and composition of the motion databases. We demonstrate the effectiveness of our approach with interactive characters that can respond to the user's control quickly while performing agile, highly dynamic movements.

[1]  Nancy S. Pollard,et al.  Responsive characters from motion fragments , 2007, SIGGRAPH 2007.

[2]  Taku Komura,et al.  Phase-functioned neural networks for character control , 2017, ACM Trans. Graph..

[3]  Zoran Popovic,et al.  Motion fields for interactive character locomotion , 2010, CACM.

[4]  Sergey Levine,et al.  DeepMimic , 2018, ACM Trans. Graph..

[5]  Taku Komura,et al.  A Deep Learning Framework for Character Motion Synthesis and Editing , 2016, ACM Trans. Graph..

[6]  Trista Pei-chun Chen,et al.  CARL , 2020, ACM Trans. Graph..

[7]  Jonas Beskow,et al.  MoGlow , 2019, ACM Trans. Graph..

[8]  Sergey Levine,et al.  Space-time planning with parameterized locomotion controllers , 2011, TOGS.

[9]  Lucas Kovar,et al.  Motion Graphs , 2002, ACM Trans. Graph..

[10]  Victor B. Zordan,et al.  Dynamic response for motion capture animation , 2005, SIGGRAPH 2005.

[11]  Jehee Lee,et al.  Synchronized multi-character motion editing , 2009, ACM Trans. Graph..

[12]  Sunmin Lee,et al.  Learning predict-and-simulate policies from unorganized human motion data , 2019, ACM Trans. Graph..

[13]  Z. Popovic,et al.  Near-optimal character animation with continuous control , 2007, ACM Trans. Graph..

[14]  Jungdam Won,et al.  Aerobatics control of flying creatures via self-regulated learning , 2018, ACM Trans. Graph..

[15]  Nancy S. Pollard,et al.  Evaluating motion graphs for character animation , 2007, TOGS.

[16]  Alla Safonova,et al.  Human Motion Synthesis with Optimization‐based Graphs , 2010, Comput. Graph. Forum.

[17]  Alla Safonova,et al.  Achieving good connectivity in motion graphs , 2008, SCA '08.

[18]  Sergey Levine,et al.  Continuous character control with low-dimensional embeddings , 2012, ACM Trans. Graph..

[19]  Libin Liu,et al.  Learning basketball dribbling skills using trajectory optimization and deep reinforcement learning , 2018, ACM Trans. Graph..

[20]  Jehee Lee,et al.  Motion Grammars for Character Animation , 2016, Comput. Graph. Forum.

[21]  Oussama Kanoun,et al.  Learned motion matching , 2020, ACM Trans. Graph..

[22]  Sebastian Starke,et al.  Neural state machine for character-scene interactions , 2019, ACM Trans. Graph..

[23]  Kyoungmin Lee,et al.  Scalable muscle-actuated human simulation and control , 2019, ACM Trans. Graph..

[24]  Jehee Lee Interactive Control of Avatars Animated with Human Motion Data , .

[25]  C. Karen Liu,et al.  Synthesis of biologically realistic human motion using joint torque actuation , 2019, ACM Trans. Graph..

[26]  Jungdam Won,et al.  A scalable approach to control diverse behaviors for physically simulated characters , 2020, ACM Trans. Graph..

[27]  J. Hodgins,et al.  Learning to Schedule Control Fragments for Physics-Based Characters Using Deep Q-Learning , 2017, ACM Trans. Graph..

[28]  Taku Komura,et al.  Simulating Multiple Character Interactions with Collaborative and Adversarial Goals , 2012, IEEE Transactions on Visualization and Computer Graphics.

[29]  J. Sheppard,et al.  Agility literature review: Classifications, training and testing , 2006, Journal of sports sciences.

[30]  Jehee Lee,et al.  Interactive character animation by learning multi-objective control , 2018, ACM Trans. Graph..

[31]  Jessica K. Hodgins,et al.  Generating and ranking diverse multi-character interactions , 2014, ACM Trans. Graph..

[32]  Taku Komura,et al.  Mode-adaptive neural networks for quadruped motion control , 2018, ACM Trans. Graph..

[33]  Michiel van de Panne,et al.  Character controllers using motion VAEs , 2020, ACM Trans. Graph..

[34]  Derek Nowrouzezahrai,et al.  Robust motion in-betweening , 2020, ACM Trans. Graph..

[35]  Jehee Lee,et al.  Motion patches: buildings blocks for virtual environments annotated with motion data , 2005, SIGGRAPH 2005.

[36]  Jessica K. Hodgins,et al.  Construction and optimal search of interpolated motion graphs , 2007, ACM Trans. Graph..

[37]  Taku Komura,et al.  Interaction patches for multi-character animation , 2008, ACM Trans. Graph..

[38]  Sebastian Starke,et al.  Local motion phases for learning multi-contact character movements , 2020, ACM Trans. Graph..