NeuroAnimator: fast neural network emulation and control of physics-based models

Animation through the numerical simulation of physicsbased graphics models offers unsurpassed realism, but it can be computationally demanding. Likewise, the search for controllers that enable physics-based models to produce desired animations usually entails formidable computational cost. This paper demonstrates the possibility of replacing the numerical simulation and control of dynamic models with a dramatically more efficient alternative. In particular, we propose the NeuroAnimator, a novel approach to creating physically realistic animation that exploits neural networks. NeuroAnimators are automatically trained off-line to emulate physical dynamics through the observation of physicsbased models in action. Depending on the model, its neural network emulator can yield physically realistic animation one or two orders of magnitude faster than conventional numerical simulation. Furthermore, by exploiting the network structure of the NeuroAnimator, we introduce a fast algorithm for learning controllers that enables either physics-based models or their neural network emulators to synthesize motions satisfying prescribed animation goals. We demonstrate NeuroAnimators for a variety of physics-based models. CR Categories: I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism—Animation; I.6.8 [Simulation and Modeling]: Types of Simulation—Animation

[1]  Frank Rosenblatt,et al.  PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMS , 1963 .

[2]  E. Polak,et al.  Computational methods in optimization : a unified approach , 1972 .

[3]  J. Meditch,et al.  Applied optimal control , 1972, IEEE Transactions on Automatic Control.

[4]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[5]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[6]  E. Catmull,et al.  A CLASS OF LOCAL INTERPOLATING SPLINES , 1974 .

[7]  Peter E. Hart,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[8]  James S. Albus,et al.  New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)1 , 1975 .

[9]  Philip E. Gill,et al.  Practical optimization , 1981 .

[10]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[11]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[12]  Richard S. Sutton,et al.  Temporal credit assignment in reinforcement learning , 1984 .

[13]  Ken Shoemake,et al.  Animating rotation with quaternion curves , 1985, SIGGRAPH.

[14]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[15]  Stephen Grossberg,et al.  Neural dynamics of adaptive sensory-motor control : ballistic eye movements , 1986 .

[16]  Jane Wilhelms,et al.  Using Dynamic Analysis for Realistic Animation of Articulated Bodies , 1987, IEEE Computer Graphics and Applications.

[17]  William H. Press,et al.  Numerical Recipes: The Art of Scientific Computing , 1987 .

[18]  Michael Kuperstein,et al.  Adaptive visual-motor coordination in multijoint robots using parallel architecture , 1987, Proceedings. 1987 IEEE International Conference on Robotics and Automation.

[19]  Michael F. Cohen,et al.  Controlling dynamic simulation with kinematic constraints , 1987, SIGGRAPH.

[20]  A. Lapedes,et al.  Nonlinear Signal Processing Using Neural Networks , 1987 .

[21]  Charles W. Anderson,et al.  Strategy Learning with Multilayer Connectionist Representations , 1987 .

[22]  John Lasseter,et al.  Principles of traditional animation applied to 3D computer animation , 1987, SIGGRAPH.

[23]  John C. Platt,et al.  Elastically deformable models , 1987, SIGGRAPH.

[24]  Gavin S. P. Miller,et al.  The motion dynamics of snakes and worms , 1988, SIGGRAPH.

[25]  Kok Lay Teo,et al.  Control parametrization: A unified approach to optimal control problems with general constraints , 1988, Autom..

[26]  Andrew P. Witkin,et al.  Spacetime constraints , 1988, SIGGRAPH.

[27]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[28]  Arun N. Netravali,et al.  Motion interpolation by optimal control , 1988, SIGGRAPH.

[29]  PAUL J. WERBOS,et al.  Generalization of backpropagation with application to a recurrent gas market model , 1988, Neural Networks.

[30]  John C. Platt,et al.  Constraints methods for flexible models , 1988, SIGGRAPH.

[31]  James K. Hahn,et al.  Realistic animation of rigid bodies , 1988, SIGGRAPH.

[32]  Mitsuo Kawato,et al.  Feedback error learning of movement by multi-layer neural network , 1988, Neural Networks.

[33]  Michael I. Jordan.  Supervised learning and systems with excess degrees of freedom , 1988 .

[34]  Ronen Barzel,et al.  A modeling system based on dynamic constraints , 1988, SIGGRAPH.

[35]  R. J. Williams,et al.  On the use of backpropagation in associative reinforcement learning , 1988, IEEE 1988 International Conference on Neural Networks.

[36]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[37]  R. Sutton,et al.  Connectionist Learning for Control: An Overview , 1989 .

[38]  B. Widrow,et al.  The truck backer-upper: an example of self-learning in neural networks , 1989, International 1989 Joint Conference on Neural Networks.

[39]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[40]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[41]  David J. Reinkensmeyer,et al.  Using associative content-addressable memories to control robots , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[42]  David Baraff,et al.  Analytical methods for dynamic simulation of non-penetrating rigid bodies , 1989, SIGGRAPH.

[43]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[44]  Demetri Terzopoulos,et al.  Physically-based facial modelling, analysis, and animation , 1990, Comput. Animat. Virtual Worlds.

[45]  Andrew P. Witkin,et al.  Fast animation and control of nonrigid structures , 1990, SIGGRAPH.

[46]  Gavin S. P. Miller,et al.  Rapid, stable fluid dynamics for computer graphics , 1990, SIGGRAPH.

[47]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[48]  Gary Ridsdale,et al.  Connectionist modelling of skill dynamics , 1990, Comput. Animat. Virtual Worlds.

[49]  David Zeltzer,et al.  Dynamic simulation of autonomous legged locomotion , 1990, SIGGRAPH.

[50]  Jessica K. Hodgins,et al.  Animation of dynamic legged locomotion , 1991, SIGGRAPH.

[51]  Arun D Kulkarni,et al.  Neural Networks for Pattern Recognition , 1991 .

[52]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[53]  R. A. Brooks,et al.  Intelligence without Representation , 1991, Artif. Intell..

[54]  Jakub Wejchert,et al.  Animation aerodynamics , 1991, SIGGRAPH.

[55]  Kumpati S. Narendra,et al.  Gradient methods for the optimization of dynamical systems containing neural networks , 1991, IEEE Trans. Neural Networks.

[56]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[57]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[58]  M G Pandy,et al.  A parameter optimization approach for the optimal control of large-scale musculoskeletal systems. , 1992, Journal of biomechanical engineering.

[59]  James C. Miller,et al.  Computer graphics principles and practice, second edition , 1992, Comput. Graph..

[60]  Michael F. Cohen,et al.  Interactive spacetime control for animation , 1992, SIGGRAPH.

[61]  L. Cooper,et al.  When Networks Disagree: Ensemble Methods for Hybrid Neural Networks , 1992 .

[62]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[63]  A. James Stewart,et al.  Beyond keyframing: an algorithmic approach to animation , 1992 .

[64]  Daniel Thalmann,et al.  Dressing animated synthetic actors with complex deformable clothes , 1992, SIGGRAPH.

[65]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[66]  Eugene Fiume,et al.  Turbulent wind fields for gaseous phenomena , 1993, SIGGRAPH.

[67]  B. Widrow,et al.  Adaptive inverse control , 1987, Proceedings of 8th IEEE International Symposium on Intelligent Control.

[68]  Michiel van de Panne,et al.  Sensor-actuator networks , 1993, SIGGRAPH.

[69]  Joe Marks,et al.  Spacetime constraints revisited , 1993, SIGGRAPH.

[70]  Zicheng Liu,et al.  Hierarchical spacetime control , 1994, SIGGRAPH.

[71]  Karl Sims,et al.  Evolving virtual creatures , 1994, SIGGRAPH.

[72]  John C. Hart,et al.  Visualizing quaternion rotation , 1994, TOGS.

[73]  Demetri Terzopoulos,et al.  Artificial fishes: physics, locomotion, perception, behavior , 1994, SIGGRAPH.

[74]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[75]  Jerry M. Mendel,et al.  Reinforcement-learning control and pattern recognition systems , 1994 .

[76]  David C. Brogan,et al.  Animating human athletics , 1995, SIGGRAPH.

[77]  Richard S. Sutton,et al.  A Menu of Designs for Reinforcement Learning Over Time , 1995 .

[78]  Mathieu Desbrun,et al.  Animating soft substances with implicit surfaces , 1995, SIGGRAPH.

[79]  Zoran Popovic,et al.  Motion warping , 1995, SIGGRAPH.

[80]  Demetri Terzopoulos,et al.  Automated learning of muscle-actuated locomotion through control abstraction , 1995, SIGGRAPH.

[81]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[82]  Ken-ichi Anjyo,et al.  Fourier principles for emotion-based human figure animation , 1995, SIGGRAPH.

[83]  Lance Williams,et al.  Motion signal processing , 1995, SIGGRAPH.

[84]  Michiel van de Panne,et al.  Motion synthesis by example , 1996 .

[85]  Michael F. Cohen,et al.  Efficient generation of motion transitions using spacetime constraints , 1996, SIGGRAPH.

[86]  Dimitris N. Metaxas,et al.  Modeling the motion of a hot, turbulent gas , 1997, SIGGRAPH.

[87]  Qinxin Yu,et al.  Synthetic motion capture for interactive virtual worlds , 1998, Proceedings Computer Animation '98 (Cat. No.98EX169).

[88]  M. Spengler.  Fast Neural Network Emulation and Control of Physics-Based Models , 1999 .