Deep Learning for Ultrasound Beamforming

Diagnostic imaging plays a critical role in healthcare, serving as a fundamental asset for timely diagnosis, disease staging and management as well as for treatment choice, planning, guidance, and follow-up. Among the diagnostic imaging options, ultrasound imaging is uniquely positioned, being a highly cost-effective modality that offers the clinician an unmatched and invaluable level of interaction, enabled by its real-time nature. Ultrasound probes are becoming increasingly compact and portable, with the market demand for low-cost pocket-sized and (in-body) miniaturized devices expanding. At the same time, there is a strong trend towards 3D imaging and the use of high-frame-rate imaging schemes; both accompanied by dramatically increasing data rates that pose a heavy burden on the probe-system communication and subsequent image reconstruction algorithms. With the demand for high-quality image reconstruction and signal extraction from less (e.g unfocused or parallel) transmissions that facilitate fast imaging, and a push towards compact probes, modern ultrasound imaging leans heavily on innovations in powerful digital receive channel processing. Beamforming, the process of mapping received ultrasound echoes to the spatial image domain, naturally lies at the heart of the ultrasound image formation chain. In this chapter on Deep Learning for Ultrasound Beamforming, we discuss why and when deep learning methods can play a compelling role in the digital beamforming pipeline, and then show how these data-driven systems can be leveraged for improved ultrasound image reconstruction.

[1]  P. P. Vaidyanathan,et al.  Maximally economic sparse arrays and cantor arrays , 2017, 2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[2]  Alycen Wiacek,et al.  CohereNet: A Deep Learning Architecture for Ultrasound Spatial Correlation Estimation and Coherence-Based Beamforming , 2020, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[3]  Marco Cuturi,et al.  Computational Optimal Transport: With Applications to Data Science , 2019 .

[4]  Bastiaan S. Veeling,et al.  Learning Sub-Sampling and Signal Recovery With Applications in Ultrasound Imaging , 2019, IEEE Transactions on Medical Imaging.

[5]  Yonina C. Eldar,et al.  Sparse Doppler Sensing Based on Nested Arrays , 2017, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[6]  Yonina C. Eldar,et al.  Innovation Rate Sampling of Pulse Streams With Application to Ultrasound Imaging , 2010, IEEE Transactions on Signal Processing.

[7]  Brett Byram,et al.  Deep neural networks for ultrasound beamforming , 2017, 2017 IEEE International Ultrasonics Symposium (IUS).

[8]  Charlie Demené,et al.  Spatiotemporal Clutter Filtering of Ultrafast Ultrasound Data Highly Increases Doppler and fUltrasound Sensitivity , 2015, IEEE Transactions on Medical Imaging.

[9]  Yonina C. Eldar,et al.  Xampling: Analog to digital at sub-Nyquist rates , 2009, IET Circuits Devices Syst..

[10]  Jørgen Jensen,et al.  Simulation of advanced ultrasound systems using Field II , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[11]  J. C. Ye,et al.  Variational Formulation of Unsupervised Deep Learning for Ultrasound Image Artifact Removal , 2021, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[12]  Yann LeCun,et al.  Learning Fast Approximations of Sparse Coding , 2010, ICML.

[13]  M. Fink,et al.  Supersonic shear imaging: a new technique for soft tissue elasticity mapping , 2004, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[14]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[15]  Jong Chul Ye,et al.  Adaptive and Compressive Beamforming Using Deep Learning for Medical Ultrasound , 2019, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[16]  T. Blumensath,et al.  Theory and Applications , 2011 .

[17]  Raoul Mallart,et al.  Adaptive focusing in scattering media through sound‐speed inhomogeneities: The van Cittert Zernike approach and focusing criterion , 1994 .

[18]  Francois Vignon,et al.  Resolution Improvement with a Fully Convolutional Neural Network Applied to Aligned Per-Channel data , 2020, 2020 IEEE International Ultrasonics Symposium (IUS).

[19]  Regev Cohen,et al.  Sparse Array Design via Fractal Geometries , 2020, IEEE Transactions on Signal Processing.

[20]  Mickael Tanter,et al.  Super-resolution Ultrasound Imaging. , 2020, Ultrasound in medicine & biology.

[21]  Pavan K. Turaga,et al.  ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  M. Tanter,et al.  Ultrafast ultrasound localization microscopy for deep super-resolution vascular imaging , 2015, Nature.

[23]  Yonina C. Eldar,et al.  Xampling in ultrasound imaging , 2011, Medical Imaging.

[24]  Yonina C. Eldar,et al.  Deep Learning for Ultrasound Image Formation: CUBDL Evaluation Framework and Open Datasets , 2021, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[25]  Brett Byram,et al.  A model and regularization scheme for ultrasonic beamforming clutter reduction , 2015, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[26]  Mickael Tanter,et al.  Ultrafast imaging in biomedical ultrasound , 2014, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[27]  Yonina C. Eldar,et al.  Xampling at the Rate of Innovation , 2011, IEEE Transactions on Signal Processing.

[28]  G. Farhat,et al.  Diagnostic ultrasound Imaging : Inside out , 2004 .

[29]  Serge J. Belongie,et al.  Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[31]  Adam C Luchies,et al.  Assessing the Robustness of Frequency-Domain Ultrasound Beamforming Using Deep Neural Networks , 2020, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[32]  Yonina C. Eldar.  Sampling Theory: Beyond Bandlimited Systems , 2015 .

[33]  Juan Esteban Arango,et al.  3D ultrafast ultrasound imaging in vivo , 2014, Physics in medicine and biology.

[34]  Purang Abolmaesumi,et al.  Cardiac point-of-care to cart-based ultrasound translation using constrained CycleGAN , 2020, International Journal of Computer Assisted Radiology and Surgery.

[35]  Sverre Holm,et al.  Wiener beamforming and the coherence factor in ultrasound imaging , 2010, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[36]  Yonina C. Eldar,et al.  Sparse Convolutional Beamforming for Ultrasound Imaging , 2018, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[37]  Yonina C. Eldar,et al.  Fourier-Domain Beamforming and Structure-Based Reconstruction for Plane-Wave Imaging , 2018, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[38]  Suhyun Park,et al.  A fast minimum variance beamforming method using principal component analysis , 2014, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[39]  Massimo Mischi,et al.  High Resolution Plane Wave Compounding Through Deep Proximal Learning , 2020, 2020 IEEE International Ultrasonics Symposium (IUS).

[40]  Yonina C. Eldar,et al.  Xampling: Signal Acquisition and Processing in Union of Subspaces , 2009, IEEE Transactions on Signal Processing.

[41]  F. Gran,et al.  Broadband minimum variance beamforming for ultrasound imaging , 2009, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[42]  Yonina C. Eldar,et al.  Sub-Nyquist Sampling and Fourier Domain Beamforming in Volumetric Ultrasound Imaging , 2015, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[43]  Yonina C. Eldar,et al.  Deep Learning in Ultrasound Imaging , 2019, Proceedings of the IEEE.

[44]  Jong Chul Ye,et al.  Efficient B-Mode Ultrasound Image Reconstruction From Sub-Sampled RF Data Using Deep Learning , 2017, IEEE Transactions on Medical Imaging.

[45]  Muyinatu A. Lediju Bell,et al.  Deep Learning to Obtain Simultaneous Image and Segmentation Outputs From a Single Input of Raw Ultrasound Channel Data , 2020, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[46]  Denis Friboulet,et al.  Compressed Sensing Reconstruction of 3D Ultrasound Data Using Dictionary Learning and Line-Wise Subsampling , 2015, IEEE Transactions on Medical Imaging.

[47]  Adam C. Luchies,et al.  Training improvements for ultrasound beamforming with deep neural networks , 2019, Physics in medicine and biology.

[48]  Yonina C. Eldar,et al.  Compressed Beamforming in Ultrasound Imaging , 2012, IEEE Transactions on Signal Processing.

[49]  Denis Friboulet,et al.  Reconstruction for Diverging-Wave Imaging Using Deep Convolutional Neural Networks , 2020, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[50]  Yonina C. Eldar,et al.  Deep-Learning Based Adaptive Ultrasound Imaging From Sub-Nyquist Channel Data , 2020, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[51]  Matthias Bo Stuart,et al.  Detection and Localization of Ultrasound Scatterers Using Convolutional Neural Networks , 2020, IEEE Transactions on Medical Imaging.

[52]  Dongwoon Hyun,et al.  Beamforming and Speckle Reduction Using Neural Networks , 2019, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[53]  B M Asl,et al.  Eigenspace-based minimum variance beamforming applied to medical ultrasound imaging , 2010, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[54]  Yonina C. Eldar,et al.  iMAP Beamforming for High-Quality High Frame Rate Imaging , 2018, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[55]  MooHo Bae,et al.  Fast Minimum Variance Beamforming Based on Legendre Polynomials , 2016, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[56]  B T Cox,et al.  k-Wave: MATLAB toolbox for the simulation and reconstruction of photoacoustic wave fields. , 2010, Journal of biomedical optics.

[57]  Yonina C. Eldar,et al.  Multichannel Sampling of Pulse Streams at the Rate of Innovation , 2010, IEEE Transactions on Signal Processing.

[58]  Jong Chul Ye,et al.  Deep Learning-based Universal Beamformer for Ultrasound Imaging , 2019, MICCAI.

[59]  M. Modat,et al.  Variational Encoder-Decoder for Joint Modality Completion and Segmentation p . In Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 , 2019 .

[60]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[61]  Yonina C. Eldar,et al.  Compressed Fourier-Domain Convolutional Beamforming for Wireless Ultrasound imaging , 2020, 2010.13171.

[62]  A. Austeng,et al.  Sparse 2-D arrays for 3-D phased array imaging - design methods , 2002, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[63]  Jean-Philippe Thiran,et al.  A deep learning approach to ultrasound image recovery , 2017, 2017 IEEE International Ultrasonics Symposium (IUS).

[64]  Yonina C. Eldar,et al.  Fourier-domain beamforming: the path to compressed ultrasound imaging , 2013, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[65]  Jong Chul Ye,et al.  Switchable Deep Beamformer , 2020, ArXiv.

[66]  Yonina C. Eldar,et al.  FoCUS: Fourier-Based Coded Ultrasound , 2017, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[67]  Jean-Philippe Thiran,et al.  A compressed beamforming framework for ultrafast ultrasound imaging , 2016, 2016 IEEE International Ultrasonics Symposium (IUS).

[68]  Yonina C. Eldar,et al.  Deep Unfolded Recovery of Sub-Nyquist Sampled Ultrasound Images , 2021, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[69]  Herbert F. Voigt,et al.  IEEE Engineering in Medicine and Biology Society , 2019, IEEE Transactions on Biomedical Engineering.