The Application of Deep Convolutional Neural Networks to Ultrasound for Modelling of Dynamic States within Human Skeletal Muscle

This paper concerns the fully automatic direct in vivo measurement of active and passive dynamic skeletal muscle states using ultrasound imaging. Despite the long standing medical need (myopathies, neuropathies, pain, injury, ageing), currently technology (electromyography, dynamometry, shear wave imaging) provides no general, non-invasive method for online estimation of skeletal intramuscular states. Ultrasound provides a technology in which static and dynamic muscle states can be observed non-invasively, yet current computational image understanding approaches are inadequate. We propose a new approach in which deep learning methods are used for understanding the content of ultrasound images of muscle in terms of its measured state. Ultrasound data synchronized with electromyography of the calf muscles, with measures of joint torque/angle were recorded from 19 healthy participants (6 female, ages: 30 ± 7.7). A segmentation algorithm previously developed by our group was applied to extract a region of interest of the medial gastrocnemius. Then a deep convolutional neural network was trained to predict the measured states (joint angle/torque, electromyography) directly from the segmented images. Results revealed for the first time that active and passive muscle states can be measured directly from standard b-mode ultrasound images, accurately predicting for a held out test participant changes in the joint angle, electromyography, and torque with as little error as 0.022°, 0.0001V, 0.256Nm (root mean square error) respectively.

[1]  K. Häkkinen,et al.  Muscle cross-sectional area and voluntary force production characteristics in elite strength- and endurance-trained athletes and sprinters , 2006, European Journal of Applied Physiology and Occupational Physiology.

[2]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[3]  Ian David Loram,et al.  Proactive Selective Inhibition Targeted at the Neck Muscles: This Proximal Constraint Facilitates Learning and Regulates Global Control , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[4]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Ian David Loram,et al.  Use of ultrasound to make noninvasive in vivo measurement of continuous changes in human muscle contractile length. , 2006, Journal of applied physiology.

[6]  Yoshua Bengio,et al.  Generative Adversarial Networks , 2014, ArXiv.

[7]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[8]  James M Wakeling,et al.  Computational methods for quantifying in vivo muscle fascicle curvature from ultrasound images. , 2011, Journal of biomechanics.

[9]  Pengfei Song,et al.  Validation of shear wave elastography in skeletal muscle. , 2013, Journal of biomechanics.

[10]  Paul Smolensky,et al.  Information processing in dynamical systems: foundations of harmony theory , 1986 .

[11]  John Darby,et al.  Automated regional analysis of B-mode ultrasound images of skeletal muscle movement , 2011, Journal of applied physiology.

[12]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[13]  Thomas Brox,et al.  FlowNet: Learning Optical Flow with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Ghassan Hamarneh,et al.  Automated Tracking of Muscle Fascicle Orientation in B-mode Ultrasound Images , 2022 .

[15]  Geoffrey E. Hinton,et al.  Recognizing Hand-written Digits Using Hierarchical Products of Experts , 2002, NIPS.

[16]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[17]  Yasuo Kawakami,et al.  Muscle Force per Cross-sectional Area is Inversely Related with Pennation Angle in Strength Trained Athletes , 2008, Journal of strength and conditioning research.

[18]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[19]  Geoffrey E. Hinton,et al.  Learning sets of filters using back-propagation , 1987 .

[20]  J. Alison Noble,et al.  Ultrasound image segmentation: a survey , 2006, IEEE Transactions on Medical Imaging.

[21]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[22]  O. Rutherford,et al.  Measurement of fibre pennation using ultrasound in the human quadriceps in vivo , 2004, European Journal of Applied Physiology and Occupational Physiology.

[23]  S. Gandevia,et al.  Measurement of muscle contraction with ultrasound imaging , 2003, Muscle & nerve.

[24]  K. Türker,et al.  Electromyography: some methodological problems and issues. , 1993, Physical therapy.

[25]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[26]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[27]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition , 2012 .

[28]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[29]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  David J. Jilk,et al.  Recurrent Processing during Object Recognition , 2011, Front. Psychol..

[31]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[32]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[33]  Kevin J. Parker,et al.  Feature-adaptive motion tracking of ultrasound image sequences using a deformable mesh , 1998, IEEE Transactions on Medical Imaging.

[34]  A. Nordez,et al.  Elastography for Muscle Biomechanics: Toward the Estimation of Individual Muscle Force , 2015, Exercise and sport sciences reviews.

[35]  Ian Loram,et al.  Real-Time Ultrasound Segmentation, Analysis and Visualisation of Deep Cervical Muscle Structure , 2017, IEEE Transactions on Medical Imaging.

[36]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[37]  Geoffrey E. Hinton,et al.  The "wake-sleep" algorithm for unsupervised neural networks. , 1995, Science.

[38]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[39]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[40]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[41]  Vasilios Baltzopoulos,et al.  Predictability of in vivo changes in pennation angle of human tibialis anterior muscle from rest to maximum isometric dorsiflexion , 1999, European Journal of Applied Physiology and Occupational Physiology.

[42]  K J Parker,et al.  Multilevel and motion model-based ultrasonic speckle tracking algorithms. , 1998, Ultrasound in medicine & biology.

[43]  John Darby,et al.  Estimating Skeletal Muscle Fascicle Curvature From B-Mode Ultrasound Image Sequences , 2013, IEEE Transactions on Biomedical Engineering.