Super-Resolution Ultrasound Localization Microscopy Through Deep Learning

Ultrasound localization microscopy has enabled super-resolution vascular imaging through precise localization of individual ultrasound contrast agents (microbubbles) across numerous imaging frames. However, analysis of high-density regions with significant overlaps among the microbubble point spread responses yields high localization errors, constraining the technique to low-concentration conditions. As such, long acquisition times are required to sufficiently cover the vascular bed. In this work, we present a fast and precise method for obtaining super-resolution vascular images from high-density contrast-enhanced ultrasound imaging data. This method, which we term Deep Ultrasound Localization Microscopy (Deep-ULM), exploits modern deep learning strategies and employs a convolutional neural network to perform localization microscopy in dense scenarios, learning the nonlinear image-domain implications of overlapping RF signals originating from such sets of closely spaced microbubbles. Deep-ULM is trained effectively using realistic on-line synthesized data, enabling robust inference in-vivo under a wide variety of imaging conditions. We show that deep learning attains super-resolution with challenging contrast-agent densities, both in-silico as well as in-vivo. Deep-ULM is suitable for real-time applications, resolving about 70 high-resolution patches (128×128 pixels) per second on a standard PC. Exploiting GPU computation, this number increases to 1250 patches per second.

[1]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.

[2]  D. Cosgrove,et al.  Angiogenesis imaging--ultrasound. , 2003, The British journal of radiology.

[3]  Kang Kim,et al.  Super-resolution ultrasound imaging method for microvasculature in vivo with a high temporal accuracy , 2018, Scientific Reports.

[4]  M. Fink,et al.  Microbubble ultrasound super-localization imaging (MUSLI) , 2011, 2011 IEEE International Ultrasonics Symposium.

[5]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[6]  F Forsberg,et al.  Ultrasound contrast agents: a review. , 1994, Ultrasound in medicine & biology.

[7]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[8]  Keith A. Lidke,et al.  Fast, single-molecule localization that achieves theoretically minimum uncertainty , 2010, Nature Methods.

[9]  Yonina C. Eldar,et al.  Sparsity-driven super-resolution in clinical contrast-enhanced ultrasound , 2017, 2017 IEEE International Ultrasonics Symposium (IUS).

[10]  Tianqi Chen,et al.  Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[11]  Yonina C. Eldar,et al.  Exploiting Flow Dynamics for Superresolution in Contrast-Enhanced Ultrasound , 2018, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[12]  C Dunsby,et al.  Acoustic super-resolution with ultrasound and microbubbles , 2013, Physics in medicine and biology.

[13]  C. Chin.  Modelling the behaviour of microbubble contrast agents for diagnostic ultrasound , 2001 .

[14]  Robert J. Eckersley,et al.  In Vivo Acoustic Super-Resolution and Super-Resolved Velocity Mapping Using Microbubbles , 2015, IEEE Transactions on Medical Imaging.

[15]  Katherine W Ferrara,et al.  Ultrasound localization microscopy to image and assess microvasculature in a rat kidney , 2017, Scientific Reports.

[16]  Foroohar Foroozan,et al.  Microbubble Localization for Three-Dimensional Superresolution Ultrasound Imaging Using Curve Fitting and Deconvolution Methods , 2018, IEEE Transactions on Biomedical Engineering.

[17]  Sevan Harput,et al.  Acoustic wave sparsely activated localization microscopy (AWSALM): Super-resolution ultrasound imaging using acoustic activation and deactivation of nanodroplets , 2018, Applied Physics Letters.

[18]  Yonina C. Eldar,et al.  Deep Learning for Super-resolution Vascular Ultrasound Imaging , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[19]  Yonina C. Eldar,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < , 2022 .

[20]  Charles Tremblay-Darveau,et al.  Sparsity-based Ultrasound Super-resolution Hemodynamic Imaging , 2017 .

[21]  Matthew Bruce,et al.  Contrast-enhanced ultrasound to visualize hemodynamic changes after rodent spinal cord injury. , 2018, Journal of neurosurgery. Spine.

[22]  Yong Cao,et al.  Three-dimensional imaging of microvasculature in the rat spinal cord following injury , 2015, Scientific Reports.

[23]  Meaghan A. O'Reilly,et al.  A super-resolution ultrasound method for brain vascular mapping. , 2013, Medical physics.

[24]  Mickael Tanter,et al.  Resolution limits of ultrafast ultrasound localization microscopy , 2015, Physics in medicine and biology.

[25]  C. Dunsby,et al.  3-D In Vitro Acoustic Super-Resolution and Super-Resolved Velocity Mapping Using Microbubbles , 2015, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[26]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[28]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Georg Schmitz,et al.  Detection and Tracking of Multiple Microbubbles in Ultrasound B-Mode Images , 2016, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[30]  Tomer Michaeli,et al.  Deep-STORM: super-resolution single-molecule microscopy by deep learning , 2018, 1801.09631.

[31]  C. Zimmer,et al.  QuickPALM: 3D real-time photoactivation nanoscopy image processing in ImageJ , 2010, Nature Methods.

[32]  Mickael Tanter,et al.  Subwavelength motion-correction for ultrafast Ultrasound Localization Microscopy , 2017, 2017 IEEE International Ultrasonics Symposium (IUS).

[33]  Fabian Kiessling,et al.  Motion model ultrasound localization microscopy for preclinical and clinical multiparametric tumor characterization , 2018, Nature Communications.

[34]  Guy M. Hagen,et al.  ThunderSTORM: a comprehensive ImageJ plug-in for PALM and STORM data analysis and super-resolution imaging , 2014, Bioinform..

[35]  Feng Lin,et al.  Low-Dose CT With a Residual Encoder-Decoder Convolutional Neural Network , 2017, IEEE Transactions on Medical Imaging.

[36]  Mickael Tanter,et al.  Sono-activated ultrasound localization microscopy , 2013 .

[37]  S. Dwivedi,et al.  Obesity May Be Bad: Compressed Convolutional Networks for Biomedical Image Segmentation , 2020 .

[38]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[39]  Tao Zhang,et al.  A Survey of Model Compression and Acceleration for Deep Neural Networks , 2017, ArXiv.

[40]  Yonina C. Eldar,et al.  SUSHI: Sparsity-Based Ultrasound Super-Resolution Hemodynamic Imaging , 2018, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[41]  M. Tanter,et al.  Ultrafast ultrasound localization microscopy for deep super-resolution vascular imaging , 2015, Nature.

[42]  Donald Reid.  An algorithm for tracking multiple targets , 1978 .

[43]  G. Schmitz,et al.  Imaging tumor vascularity by tracing single microbubbles , 2011, 2011 IEEE International Ultrasonics Symposium.

[44]  Sevan Harput,et al.  Fast Acoustic Wave Sparsely Activated Localization Microscopy: Ultrasound Super-Resolution Using Plane-Wave Activation of Nanodroplets , 2019, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[45]  R. C. Macridis.  A review , 1963 .

[46]  Charles Tremblay-Darveau,et al.  Visualizing the Tumor Microvasculature With a Nonlinear Plane-Wave Doppler Imaging Scheme Based on Amplitude Modulation , 2016, IEEE Transactions on Medical Imaging.