OpenArm 2.0: Automated Segmentation of 3D Tissue Structures for Multi-Subject Study of Muscle Deformation Dynamics

We present a novel neural-network-based pipeline for segmentation of 3D muscle and bone structures from localized 2D ultrasound data of the human arm. Building from the U-Net [1] neural network framework, we examine various data augmentation techniques and training data sets to both optimize the network’s performance on our data set and hypothesize strategies to better select training data, minimizing manual annotation time while maximizing performance. We then employ this pipeline to generate the OpenArm 2.0 data set, the first factorial set of multi-subject, multi-angle, multi-force scans of the arm with full volumetric annotation of the biceps and humerus. This data set has been made available on SimTK (https://simtk.org/projects/openarm) to enable future exploration of muscle force modeling, improved musculoskeletal graphics, and assistive device control.

[1]  Benjamin D. Robertson,et al.  Elastic ankle exoskeletons reduce soleus muscle force but not work in human hopping. , 2013, Journal of applied physiology.

[2]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[3]  G. Soulez,et al.  A 3-D ultrasound imaging robotic system to detect and quantify lower limb arterial stenoses: in vivo feasibility. , 2014, Ultrasound in medicine & biology.

[4]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[5]  G. Ettema,et al.  The moment arms of 23 muscle segments of the upper limb with varying elbow and forearm positions: Implications for motor control , 1998 .

[6]  Ayman Habib,et al.  OpenSim: Open-Source Software to Create and Analyze Dynamic Simulations of Movement , 2007, IEEE Transactions on Biomedical Engineering.

[7]  S. Delp,et al.  Upper limb muscle volumes in adult subjects. , 2007, Journal of biomechanics.

[8]  Arie E. Kaufman,et al.  Real‐time Biomechanically‐based Muscle Volume Deformation using FEM , 1998, Comput. Graph. Forum.

[9]  Vijayan K. Asari,et al.  Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation , 2018, ArXiv.

[10]  Mikel Sagardia,et al.  Ultrapiano: A novel human-machine interface applied to virtual reality , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Walter R Frontera,et al.  Relationship between force and size in human single muscle fibres , 2011, Experimental physiology.

[12]  Jessica K. Hodgins,et al.  Data-driven modeling of skin and muscle deformation , 2008, ACM Trans. Graph..

[13]  Stefan Klein,et al.  SimpleElastix: A User-Friendly, Multi-lingual Library for Medical Image Registration , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[15]  Andras Lasso,et al.  PLUS: Open-Source Toolkit for Ultrasound-Guided Intervention Systems , 2014, IEEE Transactions on Biomedical Engineering.

[16]  J. Gulati,et al.  Tonicity effects on intact single muscle fibers: relation between force and cell volume. , 1982, Science.

[17]  Max A. Viergever,et al.  Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities , 2016, MICCAI.

[18]  S. Blemker,et al.  Imaging two-dimensional displacements and strains in skeletal muscle during joint motion by cine DENSE MR. , 2008, Journal of biomechanics.

[19]  Patrice Y. Simard,et al.  Best practices for convolutional neural networks applied to visual document analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[20]  Jon G. Rokne,et al.  Modeling and simulating the deformation of human skeletal muscle based on anatomy and physiology , 2005, Comput. Animat. Virtual Worlds.

[21]  O. Schmitt.  The heat of shortening and the dynamic constants of muscle , 2017 .

[22]  K. Abdel-Malek,et al.  Knee and elbow 3D strength surfaces: peak torque-angle-velocity relationships. , 2012, Journal of applied biomechanics.

[23]  Michael Damsgaard,et al.  Analysis of musculoskeletal systems in the AnyBody Modeling System , 2006, Simulation modelling practice and theory.

[24]  Andras Lasso,et al.  Navigated Breast Tumor Excision Using Electromagnetically Tracked Ultrasound and Surgical Instruments , 2016, IEEE Transactions on Biomedical Engineering.

[25]  Ruzena Bajcsy,et al.  Empirical Quantification and Modeling of Muscle Deformation: Toward Ultrasound-Driven Assistive Device Control * This work was supported by the NSF National Robotics Initiative (award no. 81774), Siemens Healthcare (85993), and the NSF Graduate Research Fellowship Program. , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Andras Lasso,et al.  Visual aid for identifying vertebral landmarks in ultrasound , 2018, Medical Imaging.

[27]  Marcus G. Pandy,et al.  Application of ultrasound imaging to subject-specific modelling of the human musculoskeletal system , 2017 .

[28]  Andras Lasso,et al.  Open-source platforms for navigated image-guided interventions , 2016, Medical Image Anal..

[29]  Ruzena Bajcsy,et al.  Fully Automated Echocardiogram Interpretation in Clinical Practice , 2018, Circulation.

[30]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.