Deep learning segmentation of transverse musculoskeletal ultrasound images for neuromuscular disease assessment

Ultrasound imaging is a patient-friendly and robust technique for studying physiological and pathological muscles. An automatic deep learning (DL) system for the analysis of ultrasound images could be useful to support an expert operator, allowing the study of large datasets requiring less human interaction. The purpose of this study is to present a deep learning algorithm for the cross-sectional area (CSA) segmentation in transverse musculoskeletal ultrasound images, providing a quantitative grayscale analysis which is useful for studying muscles, and to validate the results in a large dataset. The dataset included 3917 images of biceps brachii, tibialis anterior and gastrocnemius medialis acquired on 1283 subjects (mean age 50 ± 21 years, 729 male). The algorithm was based on multiple deep-learning architectures, and its performance was compared to a manual expert segmentation. We compared the mean grayscale value inside the automatic and manual CSA using Bland-Altman plots and a correlation analysis. Classification in healthy and abnormal muscles between automatic and manual segmentation were compared using the grayscale value z-scores. In the test set, a Precision of 0.88 ± 0.12 and a Recall of 0.92 ± 0.09 was achieved. The network segmentation performance was slightly less in abnormal muscles, without a loss of discrimination between healthy and abnormal muscle images. Bland-Altman plots showed no clear trend in the error distribution and the two readings have a 0.99 Pearson's correlation coefficient (p < 0.001, test set). The ICC(A, 1) calculated between the z-score readings was 0.99. The algorithm achieves robust CSA segmentation performance and gives mean grayscale level information comparable to a manual operator. This could provide a helpful tool for clinicians in neuromuscular disease diagnosis and follow-up. The entire dataset and code are made available for the research community.

[1]  Filippo Molinari,et al.  Transverse Muscle Ultrasound Analysis (TRAMA): Robust and Accurate Segmentation of Muscle Cross-Sectional Area. , 2019, Ultrasound in medicine & biology.

[2]  Neil J. Joshi,et al.  Automated diagnosis of myositis from muscle ultrasound: Exploring the use of machine learning and deep learning methods , 2017, PloS one.

[3]  Young Han Lee,et al.  Artificial intelligence in musculoskeletal ultrasound imaging , 2020, Ultrasonography.

[4]  T. R. Savarimuthu,et al.  Neural networks for automatic scoring of arthritis disease activity on ultrasound images , 2019, RMD Open.

[5]  Dong Ni,et al.  Deep Learning in Medical Ultrasound Analysis: A Review , 2019, Engineering.

[6]  Theo van Walsum,et al.  Ultrasound Aided Vertebral Level Localization for Lumbar Surgery , 2017, IEEE Transactions on Medical Imaging.

[7]  Mark R Holland,et al.  Quantitative ultrasound of skeletal muscle: reliable measurements of calibrated muscle backscatter from different ultrasound systems. , 2012, Ultrasound in medicine & biology.

[8]  Davide Fontanarosa,et al.  Segmentation of Femoral Cartilage from Knee Ultrasound Images Using Mask R-CNN , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[9]  Michael D. Abràmoff,et al.  Image processing with ImageJ , 2004 .

[10]  Yiyan Song,et al.  Applying deep learning in recognizing the femoral nerve block region on ultrasound images. , 2019, Annals of translational medicine.

[11]  A. Verbeek,et al.  The Epidemiology of Neuromuscular Disorders: A Comprehensive Overview of the Literature. , 2015, Journal of neuromuscular diseases.

[12]  N. van Alfen,et al.  How useful is muscle ultrasound in the diagnostic workup of neuromuscular diseases? , 2018, Current opinion in neurology.

[13]  I-Jeng Wang,et al.  Deep embeddings for novelty detection in myopathy , 2019, Comput. Biol. Medicine.

[14]  U. Rajendra Acharya,et al.  Automated localization and segmentation techniques for B-mode ultrasound images: A review , 2018, Comput. Biol. Medicine.

[15]  Yang Chen,et al.  A Single-Shot Region-Adaptive Network for Myotendinous Junction Segmentation in Muscular Ultrasound Images , 2020, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[16]  Olivier Seynnes,et al.  Fully automated analysis of muscle architecture from B-mode ultrasound images with deep learning , 2020, ArXiv.

[17]  D Fontanarosa,et al.  Deep Learning-Based Femoral Cartilage Automatic Segmentation in Ultrasound Imaging for Guidance in Robotic Knee Arthroscopy. , 2019, Ultrasound in medicine & biology.

[18]  Nima Tajbakhsh,et al.  UNet++: A Nested U-Net Architecture for Medical Image Segmentation , 2018, DLMIA/ML-CDS@MICCAI.

[19]  M J Zwarts,et al.  Quantitative skeletal muscle ultrasonography in children with suspected neuromuscular disease , 2003, Muscle & nerve.

[20]  Filippo Molinari,et al.  Fully Automated Muscle Ultrasound Analysis (MUSA): Robust and Accurate Muscle Thickness Measurement. , 2017, Ultrasound in medicine & biology.

[21]  B. Wong,et al.  Global versus individual muscle segmentation to assess quantitative MRI-based fat fraction changes in neuromuscular diseases , 2020, European Radiology.

[22]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[23]  Yujie Li,et al.  NAS-Unet: Neural Architecture Search for Medical Image Segmentation , 2019, IEEE Access.

[24]  K. McGraw,et al.  Forming inferences about some intraclass correlation coefficients. , 1996 .

[25]  Alexey A. Shvets,et al.  Feature Pyramid Network for Multi-class Land Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[26]  N. van Alfen,et al.  Neuromuscular Ultrasound: A New Tool in Your Toolbox , 2018, Canadian Journal of Neurological Sciences / Journal Canadien des Sciences Neurologiques.

[27]  Machiel J Zwarts,et al.  Muscle ultrasound in neuromuscular disorders , 2008, Muscle & nerve.

[28]  Ingerid Reinertsen,et al.  Highlighting nerves and blood vessels for ultrasound-guided axillary nerve block procedures using neural networks , 2018, Journal of medical imaging.

[29]  Christian F. Baumgartner,et al.  Clinical evaluation of fully automated thigh muscle and adipose tissue segmentation using a U-Net deep learning architecture in context of osteoarthritic knee pain , 2019, Magnetic Resonance Materials in Physics, Biology and Medicine.

[30]  Dornoosh Zonoobi,et al.  Toward automatic diagnosis of hip dysplasia from 2D ultrasound , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[31]  Kristen M. Meiburger,et al.  Automatic segmentation of ultrasound images of gastrocnemius medialis with different echogenicity levels using convolutional neural networks , 2020, 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC).

[32]  Alexandr A. Kalinin,et al.  Albumentations: fast and flexible image augmentations , 2018, Inf..

[33]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[34]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).