Muscle‐Based Control for Character Animation

Muscle‐based control is transforming the field of physics‐based character animation through the integration of knowledge from neuroscience, biomechanics and robotics, which enhance motion realism. Since any physics‐based animation system can be extended to a muscle‐actuated system, the possibilities of growth are tremendous. However, modelling muscles and their control remains a difficult challenge. We present an organized review of over a decade of research in muscle‐based control for character animation, its fundamental concepts and future directions for development. The core of this review contains a classification of control methods, tables summarizing their key aspects and popular neuromuscular functions used within these controllers, all with the purpose of providing the reader with an overview of the field.

[1]  Jun Morimoto,et al.  Experimental Studies of a Neural Oscillator for Biped Locomotion with QRIO , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[2]  S. Hooper,et al.  Central pattern generators , 2000, Current Biology.

[3]  M G Pandy,et al.  Computer modeling and simulation of human movement. , 2001, Annual review of biomedical engineering.

[4]  Hartmut Geyer,et al.  A Muscle-Reflex Model That Encodes Principles of Legged Mechanics Produces Human Walking Dynamics and Muscle Activities , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[5]  James C. Houk,et al.  Neural Control of Muscle Length and Tension , 2011 .

[6]  Ayman Habib,et al.  OpenSim: Open-Source Software to Create and Analyze Dynamic Simulations of Movement , 2007, IEEE Transactions on Biomedical Engineering.

[7]  Hartmut Witte,et al.  ISB recommendation on definitions of joint coordinate system of various joints for the reporting of human joint motion--part I: ankle, hip, and spine. International Society of Biomechanics. , 2002, Journal of biomechanics.

[8]  A. Hill.  The heat of shortening and the dynamic constants of muscle , 1938 .

[9]  Victor B. Zordan,et al.  Breathe easy: Model and control of human respiration for computer animation , 2006, Graph. Model..

[10]  Z. Popovic,et al.  Terrain-adaptive bipedal locomotion control , 2010, ACM Trans. Graph..

[11]  F. V. D. van der Helm.  A finite element musculoskeletal model of the shoulder mechanism. , 1994, Journal of biomechanics.

[12]  Taku Komura,et al.  Creating and retargetting motion by the musculoskeletal human body model , 2000, The Visual Computer.

[13]  Victor B. Zordan,et al.  Goal-directed stepping with momentum control , 2010, SCA '10.

[14]  Eftychios Sifakis,et al.  Comprehensive biomechanical modeling and simulation of the upper body , 2009, TOGS.

[15]  J. Mizrahi,et al.  A model of fatigue and recovery in paraplegic's quadriceps muscle subjected to intermittent FES. , 1996, Journal of biomechanical engineering.

[16]  H. Ralston.  Energetics of Human Walking , 1976 .

[17]  Andrea d'Avella,et al.  Modularity for Sensorimotor Control: Evidence and a New Prediction , 2010, Journal of motor behavior.

[18]  Phil Husbands,et al.  Evolution of central pattern generators for bipedal walking in a real-time physics environment , 2002, IEEE Trans. Evol. Comput..

[19]  Christopher G. Atkeson,et al.  Constructive Incremental Learning from Only Local Information , 1998, Neural Computation.

[20]  Zoran Popovic,et al.  The space of human body shapes: reconstruction and parameterization from range scans , 2003, ACM Trans. Graph..

[21]  Dario Farina,et al.  Identifying representative synergy matrices for describing muscular activation patterns during multidirectional reaching in the horizontal plane. , 2010, Journal of neurophysiology.

[22]  Franck Plestan,et al.  Distribution of forces between synergistics and antagonistics muscles using an optimization criterion depending on muscle contraction behavior. , 2010, Journal of biomechanical engineering.

[23]  Dinesh K. Pai,et al.  Musculotendon simulation for hand animation , 2008, ACM Trans. Graph..

[24]  Torsten Reil,et al.  Biologically inspired control of physically simulated bipeds , 2001, Theory in Biosciences.

[25]  Lifeng Zhu,et al.  Adaptable Anatomical Models for Realistic Bone Motion Reconstruction , 2015, Comput. Graph. Forum.

[26]  Jack M. Winters,et al.  Modeling Musculoskeletal Movement Systems: Joint and Body Segmental Dynamics, Musculoskeletal Actuation, and Neuromuscular Control , 1990 .

[27]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[28]  Taku Komura,et al.  From Motion Capture to Real-Time Character Animation , 2008, MIG.

[29]  Michael Damsgaard,et al.  Force-dependent kinematics: a new analysis method for non-conforming joints , 2011 .

[30]  F.C.T. van der Helm,et al.  A finite element musculoskeletal model of the shoulder mechanism. , 1994 .

[31]  Geoffrey E. Hinton,et al.  NeuroAnimator: fast neural network emulation and control of physics-based models , 1998, SIGGRAPH.

[32]  Victor B. Zordan,et al.  Momentum control for balance , 2009, ACM Trans. Graph..

[33]  M. McCall,et al.  Rigid Body Dynamics , 2008 .

[34]  Jessica K. Hodgins,et al.  Biped gait transitions , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[35]  Maarten F Bobbert,et al.  Robust passive dynamics of the musculoskeletal system compensate for unexpected surface changes during human hopping. , 2009, Journal of applied physiology.

[36]  Ana Lucia Cruz Ruiz,et al.  Motion control via muscle synergies: application to throwing , 2015, MIG.

[37]  Zoran Popović,et al.  Terrain-adaptive bipedal locomotion control , 2010, SIGGRAPH 2010.

[38]  Charles Pontonnier,et al.  Inverse dynamics method using optimization techniques for the estimation of muscles forces involved in the elbow motion , 2009 .

[39]  Michael Gleicher,et al.  Retargetting motion to new characters , 1998, SIGGRAPH.

[40]  David I. W. Levin,et al.  Large-scale dynamic simulation of highly constrained strands , 2011, SIGGRAPH 2011.

[41]  M. Pandy,et al.  Dynamic optimization of human walking. , 2001, Journal of biomechanical engineering.

[42]  Walter Herzog,et al.  Model-based estimation of muscle forces exerted during movements. , 2007, Clinical biomechanics.

[43]  Victor B. Zordan,et al.  Laughing out loud: control for modeling anatomically inspired laughter using audio , 2008, SIGGRAPH Asia '08.

[44]  E Pennestrì,et al.  Virtual musculo-skeletal model for the biomechanical analysis of the upper limb. , 2007, Journal of biomechanics.

[45]  David A. Forsyth,et al.  Generalizing motion edits with Gaussian processes , 2009, ACM Trans. Graph..

[46]  N. Hogan,et al.  Does the nervous system use equilibrium-point control to guide single and multiple joint movements? , 1992, The Behavioral and brain sciences.

[47]  R Bartlett,et al.  Inverse optimization: functional and physiological considerations related to the force-sharing problem. , 1997, Critical reviews in biomedical engineering.

[48]  Duane Knudson,et al.  Fundamentals of Biomechanics , 2003, Springer US.

[49]  Weiping Li,et al.  Applied Nonlinear Control , 1991 .

[50]  Richard E. Parent,et al.  Layered construction for deformable animated characters , 1989, SIGGRAPH.

[51]  M. Damsgaard,et al.  Muscle recruitment by the min/max criterion -- a comparative numerical study. , 2001, Journal of biomechanics.

[52]  Marko Ackermann,et al.  Optimality principles for model-based prediction of human gait. , 2010, Journal of biomechanics.

[53]  Jane Wilhelms,et al.  Anatomically based modeling , 1997, SIGGRAPH.

[54]  Jessica K. Hodgins,et al.  Biped Gymnastics , 1988, Int. J. Robotics Res..

[55]  P. Leva.  Adjustments to Zatsiorsky-Seluyanov's segment inertia parameters. , 1996 .

[56]  Taesoo Kwon,et al.  Control systems for human running using an inverted pendulum model and a reference motion capture sequence , 2010, SCA '10.

[57]  Jane Wilhelms,et al.  Toward Automatic Motion Control , 1987, IEEE Computer Graphics and Applications.

[58]  Guillaume Rao,et al.  A two-step EMG-and-optimization process to estimate muscle force during dynamic movement. , 2010, Journal of biomechanics.

[59]  Gentiane Venture,et al.  Identifying musculo-tendon parameters of human body based on the musculo-skeletal dynamics computation and Hill-Stroeve muscle model , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[60]  Vladlen Koltun,et al.  Animating human lower limbs using contact-invariant optimization , 2013, ACM Trans. Graph..

[61]  R. Riemer,et al.  Uncertainties in inverse dynamics solutions: a comprehensive analysis and an application to gait. , 2008, Gait & posture.

[62]  KangKang Yin,et al.  SIMBICON: simple biped locomotion control , 2007, ACM Trans. Graph..

[63]  A. Kuo.  A least-squares estimation approach to improving the precision of inverse dynamics computations. , 1998, Journal of biomechanical engineering.

[64]  Auke Jan Ijspeert,et al.  Central pattern generators for locomotion control in animals and robots: A review , 2008, Neural Networks.

[65]  Eugene Fiume,et al.  Helping hand: an anatomically accurate inverse dynamics solution for unconstrained hand motion , 2005, SCA '05.

[66]  Demetri Terzopoulos,et al.  Heads up!: biomechanical modeling and neuromuscular control of the neck , 2006, SIGGRAPH 2006.

[67]  Luc Martin,et al.  A method to combine numerical optimization and EMG data for the estimation of joint moments under dynamic conditions. , 2004, Journal of biomechanics.

[68]  Victor Ng-Thow-Hing,et al.  Dynamic Animation and Control Environment , 2005, Graphics Interface.

[69]  F.E. Zajac,et al.  An interactive graphics-based model of the lower extremity to study orthopaedic surgical procedures , 1990, IEEE Transactions on Biomedical Engineering.

[70]  Demetri Terzopoulos,et al.  Artificial fishes: physics, locomotion, perception, behavior , 1994, SIGGRAPH.

[71]  Dinesh K. Pai,et al.  Active volumetric musculoskeletal systems , 2014, ACM Trans. Graph..

[72]  Mikhail Fain,et al.  Biomechanical simulation and control of hands and tendinous systems , 2015, ACM Trans. Graph..

[73]  Kazunori Hase,et al.  Human gait simulation with a neuromusculoskeletal model and evolutionary computation , 2003, Comput. Animat. Virtual Worlds.

[74]  J. Challis,et al.  Individual Sarcomere Lengths in Whole Muscle Fibers and Optimal Fiber Length Computation , 2010, Anatomical record.

[75]  H. Hatze,et al.  A myocybernetic control model of skeletal muscle , 1977, Biological Cybernetics.

[76]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[77]  Andrew P. Witkin,et al.  Spacetime constraints , 1988, SIGGRAPH.

[78]  Jessica K. Hodgins,et al.  Capturing and animating skin deformation in human motion , 2006, SIGGRAPH '06.

[79]  Georges Dumont,et al.  Strengths and limitations of a musculoskeletal model for an analysis of simulated meat cutting tasks. , 2014, Applied ergonomics.

[80]  M. Pandy,et al.  A Dynamic Optimization Solution for Vertical Jumping in Three Dimensions. , 1999, Computer methods in biomechanics and biomedical engineering.

[81]  David G Lloyd,et al.  Neuromusculoskeletal modeling: estimation of muscle forces and joint moments and movements from measurements of neural command. , 2004, Journal of applied biomechanics.

[82]  Matthew Millard,et al.  A Computationally Efficient Muscle Model , 2012 .

[83]  Sybert H. Stroeve,et al.  Learning combined feedback and feedforward control of a musculoskeletal system , 1996, Biological Cybernetics.

[84]  Michael Damsgaard,et al.  Analysis of musculoskeletal systems in the AnyBody Modeling System , 2006, Simul. Model. Pract. Theory.

[85]  O. Schmitt.  The heat of shortening and the dynamic constants of muscle , 2017 .

[86]  Scott L. Hooper.  Central Pattern Generators , 2001 .

[87]  A. Karpathy,et al.  Locomotion skills for simulated quadrupeds , 2011, ACM Trans. Graph..

[88]  J. Doyle,et al.  Robust and optimal control , 1995, Proceedings of 35th IEEE Conference on Decision and Control.

[89]  Victor B. Zordan,et al.  Laughing out loud: control for modeling anatomically inspired laughter using audio , 2008, SIGGRAPH 2008.

[90]  Gentiane Venture,et al.  APPLICATION OF NON-LINEAR LEAST SQUARE METHOD TO ESTIMATE THE MUSCLE DYNAMICS OF THE ELBOW JOINT , 2006 .

[91]  Dieter Rosenbaum,et al.  on definitions of joint coordinate system of various joints for the reporting of human joint motion — part I : ankle , hip , and spine , 2002 .

[92]  Jovan Popovic,et al.  Multiobjective control with frictional contacts , 2007, SCA '07.

[93]  Zoran Popovic,et al.  Discovery of complex behaviors through contact-invariant optimization , 2012, ACM Trans. Graph..

[94]  Singiresu S. Rao.  Engineering Optimization : Theory and Practice , 2010 .

[95]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[96]  M. Ceruso.  Clinical Mechanics of the Hand, 3rd Edition , 2001 .

[97]  Reinhard Blickhan,et al.  Positive force feedback in bouncing gaits? , 2003, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[98]  Gavin S. P. Miller,et al.  The motion dynamics of snakes and worms , 1988, SIGGRAPH.

[99]  R. Edwards,et al.  Human muscle function and fatigue. , 2008, Ciba Foundation symposium.

[100]  Antonie J. van den Bogert,et al.  A real-time system for biomechanical analysis of human movement and muscle function , 2013, Medical & Biological Engineering & Computing.

[101]  Martin de Lasa,et al.  Robust physics-based locomotion using low-dimensional planning , 2010, ACM Trans. Graph..

[102]  A. Prochazka,et al.  Positive force feedback control of muscles. , 1997, Journal of neurophysiology.

[103]  C Pontonnier,et al.  A bio-inspired limb controller for avatar animation , 2014, Computer methods in biomechanics and biomedical engineering.

[104]  C. Karen Liu,et al.  Soft body locomotion , 2012, ACM Trans. Graph..

[105]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[106]  L. Chèze,et al.  Adjustments to McConville et al. and Young et al. body segment inertial parameters. , 2007, Journal of Biomechanics.

[107]  Demetri Terzopoulos,et al.  Heads up!: biomechanical modeling and neuromuscular control of the neck , 2006, ACM Trans. Graph..

[108]  Christoph Bregler,et al.  Motion capture assisted animation: texturing and synthesis , 2002, ACM Trans. Graph..

[109]  R. Crowninshield,et al.  A physiologically based criterion of muscle force prediction in locomotion. , 1981, Journal of biomechanics.

[110]  J. Challis.  Producing physiologically realistic individual muscle force estimations by imposing constraints when using optimization techniques. , 1997, Medical engineering & physics.

[111]  E. Fiume,et al.  A Survey of Modeling and Simulation of Skeletal Muscle , 2010 .

[112]  Roy Featherstone,et al.  Rigid Body Dynamics Algorithms , 2007 .

[113]  P. Helm.  Clinical mechanics of the hand , 1986 .

[114]  Demetri Terzopoulos,et al.  Automated learning of muscle-actuated locomotion through control abstraction , 1995, SIGGRAPH.

[115]  Taesoo Kwon,et al.  Locomotion control for many-muscle humanoids , 2014, ACM Trans. Graph..

[116]  Michiel van de Panne,et al.  Flexible muscle-based locomotion for bipedal creatures , 2013, ACM Trans. Graph..

[117]  C P Tsui,et al.  A 3D skeletal muscle model coupled with active contraction of muscle fibres and hyperelastic behaviour. , 2009, Journal of biomechanics.

[118]  Eftychios Sifakis,et al.  Realistic Biomechanical Simulation and Control of Human Swimming , 2014, ACM Trans. Graph..

[119]  Vladlen Koltun,et al.  Optimizing locomotion controllers using biologically-based actuators and objectives , 2012, ACM Trans. Graph..

[120]  Shinya Aoi,et al.  Locomotion Control of a Biped Robot Using Nonlinear Oscillators , 2005, Auton. Robots.

[121]  Eugene Fiume,et al.  Anatomically-based models for physical and geometric reconstruction of humans and other animals , 2001 .

[122]  A. G. Feldman.  Once More on the Equilibrium-Point Hypothesis (λ Model) for Motor Control , 1986 .

[123]  Maarten F. Bobbert,et al.  The contribution of muscle properties in the control of explosive movements , 1993, Biological Cybernetics.

[124]  Stefano Panzeri,et al.  Muscle synergies in neuroscience and robotics: from input-space to task-space perspectives , 2013, Front. Comput. Neurosci..

[125]  Taku Komura,et al.  A Muscle‐based Feed‐forward Controller of the Human Body , 1997, Comput. Graph. Forum.

[126]  A. J. van den Bogert,et al.  Intrinsic muscle properties facilitate locomotor control - a computer simulation study. , 1998, Motor control.

[127]  W. T. Dempster,et al.  SPACE REQUIREMENTS OF THE SEATED OPERATOR, GEOMETRICAL, KINEMATIC, AND MECHANICAL ASPECTS OF THE BODY WITH SPECIAL REFERENCE TO THE LIMBS , 1955 .

[128]  David A. Winter,et al.  Biomechanics and Motor Control of Human Movement , 1990 .

[129]  Michiel van de Panne,et al.  Sensor-actuator networks , 1993, SIGGRAPH.

[130]  Karl Sims,et al.  Evolving virtual creatures , 1994, SIGGRAPH.

[131]  Paul W. Brand,et al.  Clinical mechanics of the hand , 1985 .

[132]  Georges Dumont,et al.  Dynamics-based analysis and synthesis of human locomotion , 2007, The Visual Computer.

[133]  Jessica K. Hodgins,et al.  Capturing and animating skin deformation in human motion , 2006, SIGGRAPH 2006.

[134]  Yoonsang Lee,et al.  Data-driven biped control , 2010, ACM Trans. Graph..

[135]  Steven Dubowsky,et al.  A coordinated Jacobian transpose control for mobile multi-limbed robotic systems , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[136]  L. Ting,et al.  Muscle synergies characterizing human postural responses. , 2007, Journal of neurophysiology.

[137]  F. Zajac.  Muscle and tendon: properties, models, scaling, and application to biomechanics and motor control. , 1989, Critical reviews in biomedical engineering.

[138]  Demetri Terzopoulos,et al.  Biomechanical modeling and control of the human body for computer animation , 2008 .

[139]  Gentaro Taga,et al.  A model of the neuro-musculo-skeletal system for human locomotion , 1995, Biological Cybernetics.

[140]  John Rasmussen,et al.  Functional Scaling of Musculoskeletal Models , 2011 .

[141]  Jessica K. Hodgins,et al.  Simulating leaping, tumbling, landing and balancing humans , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[142]  Hans-Peter Seidel,et al.  Eurographics/siggraph Symposium on Computer Animation (2003) Construction and Animation of Anatomically Based Human Hand Models , 2022 .

[143]  S. Delp,et al.  Image‐based musculoskeletal modeling: Applications, advances, and future opportunities , 2007, Journal of magnetic resonance imaging : JMRI.

[144]  M. van de Panne,et al.  Generalized biped walking control , 2010, ACM Trans. Graph..

[145]  H F J M Koopman,et al.  Morphological muscle and joint parameters for musculoskeletal modelling of the lower extremity. , 2005, Clinical biomechanics.

[146]  Antonie J van den Bogert,et al.  A weighted least squares method for inverse dynamic analysis , 2008, Computer methods in biomechanics and biomedical engineering.

[147]  K N An,et al.  Determination of muscle and joint forces: a new technique to solve the indeterminate problem. , 1984, Journal of biomechanical engineering.

[148]  Qiang Huang,et al.  Sensory reflex control for humanoid walking , 2005, IEEE Transactions on Robotics.

[149]  Hiroshi Shimizu,et al.  Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment , 1991, Biological Cybernetics.

[150]  J. Mizrahi,et al.  A musculotendon model of the fatigue profiles of paralyzed quadriceps muscle under FES , 1993, IEEE Transactions on Biomedical Engineering.

[151]  Gentaro Taga,et al.  A model of the neuro-musculo-skeletal system for human locomotion , 1995, Biological Cybernetics.

[152]  Gentaro Taga,et al.  A model of the neuro-musculo-skeletal system for anticipatory adjustment of human locomotion during obstacle avoidance , 1998, Biological Cybernetics.

[153]  Torsten Bumgarner,et al.  Biomechanics and Motor Control of Human Movement , 2013 .

[154]  C Pontonnier,et al.  Identifying representative muscle synergies in overhead football throws , 2015, Computer methods in biomechanics and biomedical engineering.

[155]  Michael Neff,et al.  Modeling tension and relaxation for computer animation , 2002, SCA '02.

[156]  Nicolas Pronost,et al.  Interactive Character Animation Using Simulated Physics: A State‐of‐the‐Art Review , 2012, Comput. Graph. Forum.

[157]  Emilio Bizzi,et al.  Combinations of muscle synergies in the construction of a natural motor behavior , 2003, Nature Neuroscience.

[158]  Timothée Masquelier,et al.  Neural variability, or lack thereof , 2013, Front. Comput. Neurosci..

[159]  Weiguang Si,et al.  Realistic Simulation and Control of Human Swimming and Underwater Movement , 2013 .

[160]  M. Vukobratovic,et al.  Biped Locomotion , 1990 .

[161]  Bryan Buchholz,et al.  ISB recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion--Part II: shoulder, elbow, wrist and hand. , 2005, Journal of biomechanics.

[162]  Daniel Thalmann,et al.  Fast realistic human body deformations for animation and VR applications , 1996, Proceedings of CG International '96.

[163]  Ronald Fedkiw,et al.  Creating and simulating skeletal muscle from the visible human data set , 2005, IEEE Transactions on Visualization and Computer Graphics.

[164]  Shinya Aoi,et al.  Stability analysis of a simple walking model driven by an oscillator with a phase reset using sensory feedback , 2006, IEEE Transactions on Robotics.

[165]  Jack M. Winters,et al.  Biomechanics and Neural Control of Posture and Movement , 2011, Springer New York.