Ultrasound Scatterer Density Classification Using Convolutional Neural Networks and Patch Statistics

Quantitative ultrasound (QUS) can reveal crucial information on tissue properties, such as scatterer density. If the scatterer density per resolution cell is above or below 10, the tissue is considered as fully developed speckle (FDS) or underdeveloped speckle (UDS), respectively. Conventionally, the scatterer density has been classified using estimated statistical parameters of the amplitude of backscattered echoes. However, if the patch size is small, the estimation is not accurate. These parameters are also highly dependent on imaging settings. In this article, we adapt convolutional neural network (CNN) architectures for QUS and train them using simulation data. We further improve the network’s performance by utilizing patch statistics as additional input channels. Inspired by deep supervision and multitask learning, we propose a second method to exploit patch statistics. We evaluate the networks using simulation data and experimental phantoms. We also compare our proposed methods with different classic and deep learning models and demonstrate their superior performance in the classification of tissues with different scatterer density values. The results also show that we are able to classify scatterer density in different imaging parameters with no need for a reference phantom. This work demonstrates the potential of CNNs in classifying scatterer density in ultrasound images.

[1]  B. Goldberg,et al.  Classification of ultrasonic B-mode images of breast masses using Nakagami distribution , 2001, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[2]  J. Arendt Paper presented at the 10th Nordic-Baltic Conference on Biomedical Imaging: Field: A Program for Simulating Ultrasound Systems , 1996 .

[3]  Jianwen Luo,et al.  Learning the implicit strain reconstruction in ultrasound elastography using privileged information , 2019, Medical Image Anal..

[4]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Robert Rohling,et al.  A Spatially Weighted Regularization Method for Attenuation Coefficient Estimation , 2019, 2019 IEEE International Ultrasonics Symposium (IUS).

[6]  Timothy J. Hall,et al.  Analysis of Coherent and Diffuse Scattering Using a Reference Phantom , 2016, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[7]  Chien-Cheng Chang,et al.  Performance Evaluation of Ultrasonic Nakagami Image in Tissue Characterization , 2008, Ultrasonic imaging.

[8]  U. Rajendra Acharya,et al.  Automated localization and segmentation techniques for B-mode ultrasound images: A review , 2018, Comput. Biol. Medicine.

[9]  Chien-Cheng Chang,et al.  Classification of breast masses by ultrasonic Nakagami imaging: a feasibility study , 2008, Physics in medicine and biology.

[10]  Roberto J. Lavarello,et al.  In Vivo Estimation of Attenuation and Backscatter Coefficients From Human Thyroids , 2016, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[11]  Tomy Varghese,et al.  Scatterer number density considerations in reference phantom-based attenuation estimation. , 2014, Ultrasound in medicine & biology.

[12]  Guy Cloutier,et al.  Estimation Method of the Homodyned K-Distribution Based on the Mean Intensity and Two Log-Moments , 2013, SIAM J. Imaging Sci..

[13]  Orcun Goksel,et al.  Deep Network for Scatterer Distribution Estimation for Ultrasound Image Simulation , 2020, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[14]  Leslie N. Smith,et al.  Cyclical Learning Rates for Training Neural Networks , 2015, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[15]  F. Foster,et al.  Non-Gaussian statistics and temporal variations of the ultrasound signal backscattered by blood at frequencies between 10 and 58 MHz. , 2004, The Journal of the Acoustical Society of America.

[16]  Jonathan Mamou,et al.  Quantitative Ultrasound in Soft Tissues , 2013, Springer Netherlands.

[17]  S. Ima-Nirwana,et al.  Calcaneal Quantitative Ultrasound as a Determinant of Bone Health Status: What Properties of Bone Does It Reflect? , 2013, International journal of medical sciences.

[18]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.

[19]  Hassan Rivaz,et al.  Two-stage ultrasound image segmentation using U-Net and test time augmentation , 2020, International Journal of Computer Assisted Radiology and Surgery.

[20]  G. Fichtinger,et al.  P3E-9 Ultrasound Speckle Detection Using Low Order Moments , 2006, 2006 IEEE Ultrasonics Symposium.

[21]  Zhuowen Tu,et al.  Training Deeper Convolutional Networks with Deep Supervision , 2015, ArXiv.

[22]  Andrzej Nowicki,et al.  Combining Nakagami imaging and convolutional neural networks for breast lesion classification , 2017, 2017 IEEE International Ultrasonics Symposium (IUS).

[23]  Samuel Mikaelian,et al.  On the statistics of ultrasonic spectral parameters. , 2006, Ultrasound in medicine & biology.

[24]  Hassan Rivaz,et al.  Segmentation of Ultrasound Images based on Scatterer Density using U-Net , 2020, 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC).

[25]  Ying-Hsiu Lin,et al.  Effect of ultrasound frequency on the Nakagami statistics of human liver tissues , 2017, PloS one.

[26]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[27]  R. F. Wagner,et al.  Statistics of Speckle in Ultrasound B-Scans , 1983, IEEE Transactions on Sonics and Ultrasonics.

[28]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Ivan M. Rosado-Mendez,et al.  Advanced Spectral Analysis Methods for Quantification of Coherent Ultrasound Scattering: Applications in the Breast , 2014 .

[30]  Chih-Chung Huang,et al.  Characterization of lamina propria and vocal muscle in human vocal fold tissue by ultrasound Nakagami imaging. , 2011, Medical physics.

[31]  Chien-Cheng Chang,et al.  Using ultrasound Nakagami imaging to assess liver fibrosis in rats. , 2012, Ultrasonics.

[32]  K. Boone,et al.  Effect of skin impedance on image quality and variability in electrical impedance tomography: a model study , 1996, Medical and Biological Engineering and Computing.

[33]  Richard James Housden,et al.  Sensorless freehand 3D ultrasound in real tissue: Speckle decorrelation without fully developed speckle , 2006, Medical Image Anal..

[34]  P. Fitzgerald,et al.  Non-Rayleigh first-order statistics of ultrasonic backscatter from normal myocardium. , 1993, Ultrasound in medicine & biology.

[35]  J F Greenleaf,et al.  Speckle analysis using signal to noise ratios based on fractional order moments. , 1995, Ultrasonic imaging.

[36]  Aya Kamaya,et al.  Quantitative ultrasound approaches for diagnosis and monitoring hepatic steatosis in nonalcoholic fatty liver disease , 2020, Theranostics.

[37]  R. F. Wagner,et al.  Describing small-scale structure in random media using pulse-echo ultrasound. , 1990, The Journal of the Acoustical Society of America.

[38]  Qiang Yang,et al.  An Overview of Multi-task Learning , 2018 .

[39]  Andrew H. Gee,et al.  Speckle detection in ultrasound images using first order statistics , 2001 .

[40]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[41]  G. Cloutier,et al.  Diagnostic Accuracy of Echo Envelope Statistical Modeling Compared to B‐Mode and Power Doppler Ultrasound Imaging in Patients With Clinically Diagnosed Lateral Epicondylosis of the Elbow , 2019, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[42]  L. X. Yao,et al.  Backscatter Coefficient Measurements Using a Reference Phantom to Extract Depth-Dependent Instrumentation Factors , 1990, Ultrasonic imaging.

[43]  Len Du How Much Deep Learning does Neural Style Transfer Really Need? An Ablation Study , 2022 .

[44]  Hassan Rivaz,et al.  Displacement Estimation in Ultrasound Elastography Using Pyramidal Convolutional Neural Network , 2020, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[45]  Timothy J Hall,et al.  Simultaneous backscatter and attenuation estimation using a least squares method with constraints. , 2011, Ultrasound in medicine & biology.

[46]  Amir Asif,et al.  Multi-Focus Ultrasound Imaging Using Generative Adversarial Networks , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[47]  Yi Wang,et al.  Deeply-Supervised Networks With Threshold Loss for Cancer Detection in Automated Breast Ultrasound , 2020, IEEE Transactions on Medical Imaging.

[48]  G. Fichtinger,et al.  9C-1 Beam Steering Approach for Speckle Characterization and Out-of-Plane Motion Estimation in Real Tissue , 2007, 2007 IEEE Ultrasonics Symposium Proceedings.

[49]  J. Ophir,et al.  Elastography: A Quantitative Method for Imaging the Elasticity of Biological Tissues , 1991, Ultrasonic imaging.

[50]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[52]  Olivier Bernard,et al.  A Pilot Study on Convolutional Neural Networks for Motion Estimation From Ultrasound Images , 2020, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[53]  Hassan Rivaz,et al.  Low Variance Estimation of Backscatter Quantitative Ultrasound Parameters Using Dynamic Programming , 2018, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[54]  E. Jakeman Speckle Statistics With A Small Number Of Scatterers , 1984 .

[55]  Jiaying Liu,et al.  Adaptive Batch Normalization for practical domain adaptation , 2018, Pattern Recognit..

[56]  Helmut Ermert,et al.  An ultrasound research interface for a clinical system. , 2006, IEEE transactions on ultrasonics, ferroelectrics, and frequency control.

[57]  J Alison Noble,et al.  Modeling of errors in Nakagami imaging: illustration on breast mass characterization. , 2014, Ultrasound in medicine & biology.

[58]  Claes Lundström,et al.  A Closer Look at Domain Shift for Deep Learning in Histopathology , 2019, ArXiv.

[59]  K. Parker,et al.  Deviations from Rayleigh Statistics in Ultrasonic Speckle , 1988, Ultrasonic imaging.

[60]  Yonina C. Eldar,et al.  Deep Learning for Super-resolution Vascular Ultrasound Imaging , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[61]  Wen-Hung Kuo,et al.  Small-window parametric imaging based on information entropy for ultrasound tissue characterization , 2017, Scientific Reports.

[62]  Hassan Rivaz,et al.  A Pilot Study on Scatterer Density Classification of Ultrasound Images Using Deep Neural Networks , 2020, 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC).

[63]  Jonathan Mamou,et al.  Review of Quantitative Ultrasound: Envelope Statistics and Backscatter Coefficient Imaging and Contributions to Diagnostic Ultrasound , 2016, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[64]  W. Youden,et al.  Index for rating diagnostic tests , 1950, Cancer.

[65]  Michal Byra,et al.  Combining Nakagami imaging and convolutional neural network for breast lesion classification , 2017, 2017 IEEE International Ultrasonics Symposium (IUS).

[66]  Richard W Prager,et al.  Analysis of speckle in ultrasound images using fractional order statistics and the homodyned k-distribution. , 2002, Ultrasonics.

[67]  Debabrata Ghosh,et al.  Deep Learning of Spatiotemporal Filtering for Fast Super-Resolution Ultrasound Imaging , 2020, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[68]  H. Rivaz Ultrasound Speckle Detection Using Low Order Moments , 2009 .