Analyzing Muscle Activity and Force with Skin Shape Captured by Non-contact Visual Sensor

Estimating physical information by vision as humans do is useful for the applications with physical interaction in the real world. For example, observing muscle bulging infers how much force a person puts on the muscle to interact with an object or environment. Since the human skin deforms due to muscle activity, it is expected that skin deformation gives information to analyze human motion. This paper demonstrates that biomechanical information can be derived from skin shape by analyzing the relationship between skin deformation, force produced by muscles, and muscle activity. We first obtained the dataset simultaneously acquired by a range sensor, a force sensor, and electromyograph EMG sensors. Since recent range sensors based on non-contact visual measurement acquires accurate and dense shape of an object at high frame rate, the deforming skin can be observed. The deformation is calculated by finding the correspondence between a template shape and each range scan. The relationship between skin deformation and other data is learned. In this paper, the following problems are considered: 1 estimating force from skin shape, 2 estimating muscle activity from skin shape, 3 synthesizing skin shape from muscle activity. In the experiments, the database learned from the sensor data can be used for the above problems, and the skin shape gives useful information to explain the muscle activity.

[1]  Marcus A. Magnor,et al.  Capture and Statistical Modeling of Arm‐Muscle Deformations , 2013, Comput. Graph. Forum.

[2]  Sami Romdhani,et al.  Optimal Step Nonrigid ICP Algorithms for Surface Registration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Huamin Wang,et al.  Modeling deformable objects from a single depth camera , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[4]  Luc Van Gool,et al.  Face/Off: live facial puppetry , 2009, SCA '09.

[5]  Olga Sorkine-Hornung,et al.  Template-Based 3D Model Fitting Using Dual-Domain Relaxation , 2011, IEEE Transactions on Visualization and Computer Graphics.

[6]  Christian Theobalt,et al.  Capture of arm-muscle deformations using a depth-camera , 2013, CVMP '13.

[7]  Li Zhang,et al.  Spacetime stereo: shape recovery for dynamic scenes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[8]  Darius Burschka,et al.  Deformable 3D Shape Registration Based on Local Similarity Transforms , 2011, Comput. Graph. Forum.

[9]  Jovan Popović,et al.  Deformation transfer for triangle meshes , 2004, SIGGRAPH 2004.

[10]  Zoran Popovic,et al.  The space of human body shapes: reconstruction and parameterization from range scans , 2003, ACM Trans. Graph..

[11]  Leonidas J. Guibas,et al.  Non-Rigid Registration Under Isometric Deformations , 2008 .

[12]  H. Seidel,et al.  Isometric registration of ambiguous and partial data , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Hao Li,et al.  Global Correspondence Optimization for Non‐Rigid Registration of Depth Scans , 2008, Comput. Graph. Forum.

[14]  André Oosterlinck,et al.  Range Image Acquisition with a Single Binary-Encoded Light Pattern , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Eiichi Yoshida,et al.  As‐Conformal‐As‐Possible Surface Registration , 2014, Comput. Graph. Forum.

[16]  Luc Van Gool,et al.  Fast 3D Scanning with Automatic Motion Compensation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[18]  Luc Van Gool,et al.  Hough Forests for Object Detection, Tracking, and Action Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Vladimir Medved,et al.  Standards for Reporting EMG Data , 2000, Journal of Electromyography and Kinesiology.

[20]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH 2005.

[21]  Jessica K. Hodgins,et al.  Data-driven modeling of skin and muscle deformation , 2008, SIGGRAPH 2008.

[22]  Ken Jackson,et al.  Modeling and Simulation of Skeletal Muscle for Computer Graphics: A Survey , 2012, Found. Trends Comput. Graph. Vis..

[23]  J. Shotton,et al.  Decision Forests for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning , 2011 .

[24]  Ryuzo Okada,et al.  Discriminative generalized hough transform for object dectection , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[25]  Yasushi Yagi,et al.  Grid-Based Active Stereo with Single-Colored Wave Pattern for Dense One-shot 3D Scan , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.