Musculoskeletal-see-through mirror: computational modeling and algorithm for whole-body muscle activity visualization in real time.

In this paper, we present a system that estimates and visualizes muscle tensions in real time using optical motion capture and electromyography (EMG). The system overlays rendered musculoskeletal human model on top of a live video image of the subject. The subject therefore has an impression that he/she sees the muscles with tension information through the cloth and skin. The main technical challenge lies in real-time estimation of muscle tension. Since existing algorithms using mathematical optimization to distribute joint torques to muscle tensions are too slow for our purpose, we develop a new algorithm that computes a reasonable approximation of muscle tensions based on the internal connections between muscles known as neuronal binding. The algorithm can estimate the tensions of 274 muscles in only 16 ms, and the whole visualization system runs at about 15 fps. The developed system is applied to assisting sport training, and the user case studies show its usefulness. Possible applications include interfaces for assisting rehabilitation.

[1]  Scott L. Delp,et al.  A computational framework for simulating and analyzing human and animal movement , 2000, Comput. Sci. Eng..

[2]  Vladimir Medved,et al.  Standards for Reporting EMG Data , 2000, Journal of Electromyography and Kinesiology.

[3]  Ronald Fedkiw,et al.  Creating and simulating skeletal muscle from the visible human data set , 2005, IEEE Transactions on Visualization and Computer Graphics.

[4]  Daniel M. Wolpert,et al.  Making smooth moves , 2022 .

[5]  Jessica K. Hodgins,et al.  Data-driven modeling of skin and muscle deformation , 2008, SIGGRAPH 2008.

[6]  T. Ohtsuki,et al.  Muscle activity and accuracy of performance of the smash stroke in badminton with reference to skill and practice , 2000, Journal of sports sciences.

[7]  I. Engberg,et al.  An electromyographic analysis of muscular activity in the hindlimb of the cat during unrestrained locomotion. , 1969, Acta physiologica Scandinavica.

[8]  Y. Nakamura,et al.  Somatosensory computation for man-machine interface from motion-capture data and musculoskeletal human model , 2005, IEEE Transactions on Robotics.

[9]  D B Chaffin,et al.  Lumbar muscle force estimation using a subject-invariant 5-parameter EMG-based model. , 1998, Journal of biomechanics.

[10]  Katsu Yamane,et al.  Computationally fast estimation of muscle tension for realtime Bio-feedback , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[11]  Robert J. Wood,et al.  Towards a 3g crawling robot through the integration of microrobot technologies , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[12]  M. Illert,et al.  Pattern of monosynaptic Ia connections in the cat forelimb. , 1989, The Journal of physiology.

[13]  M. Knott,et al.  Proprioceptive Neuromuscular Facilitation: Patterns and Techniques , 1957 .

[14]  Katsu Yamane,et al.  Estimation of Physically and Physiologically Valid Somatosensory Information , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[15]  Y. Nakamura,et al.  Macroscopic Modeling and Identification of the Human Neuromuscular Network , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[16]  M. Damsgaard,et al.  Muscle recruitment by the min/max criterion -- a comparative numerical study. , 2001, Journal of biomechanics.

[17]  Denis Zorin,et al.  Real-time rendering of textures with feature curves , 2008, TOGS.

[18]  D. Lloyd,et al.  An EMG-driven musculoskeletal model to estimate muscle forces and knee joint moments in vivo. , 2003, Journal of biomechanics.

[19]  Sybert H. Stroeve,et al.  Impedance characteristics of a neuromusculoskeletal model of the human arm I. Posture control , 1999, Biological Cybernetics.

[20]  Ayman Habib,et al.  OpenSim: Open-Source Software to Create and Analyze Dynamic Simulations of Movement , 2007, IEEE Transactions on Biomedical Engineering.

[21]  Yoshihiko Nakamura,et al.  Inverse kinematic solutions with singularity robustness for robot manipulator control , 1986 .

[22]  David E. Orin,et al.  Kinematic and kinetic analysis of open-chain linkages utilizing Newton-Euler methods , 1979 .

[23]  F. Zajac.  Muscle and tendon: properties, models, scaling, and application to biomechanics and motor control. , 1989, Critical reviews in biomedical engineering.

[24]  M Illert,et al.  Monosynaptic Ia pathways and motor behaviour of the cat distal forelimb. , 1996, Acta neurobiologiae experimentalis.

[25]  M. Illert,et al.  Monosynaptic Ia pathways at the cat shoulder , 1999, The Journal of physiology.

[26]  E. Forster,et al.  Extension of a state-of-the-art optimization criterion to predict co-contraction. , 2004, Journal of biomechanics.

[27]  C. Patel,et al.  RANDOMISED CONTROLLED TRIAL OF YOGA AND BIO-FEEDBACK IN MANAGEMENT OF HYPERTENSION , 1975, The Lancet.

[28]  M. Pandy,et al.  A phenomenological model for estimating metabolic energy consumption in muscle contraction. , 2004, Journal of biomechanics.

[29]  S. Delp,et al.  Muscular resistance to varus and valgus loads at the elbow. , 1998, Journal of biomechanical engineering.