Ultrasound image reconstruction from plane wave radio-frequency data by self-supervised deep neural network

Image reconstruction from radio-frequency (RF) data is crucial for ultrafast plane wave ultrasound (PWUS) imaging. Compared with the traditional delay-and-sum (DAS) method based on relatively imprecise assumptions, sparse regularization (SR) method directly solves the inverse problem of image reconstruction and has presented significant improvement in the image quality when the frame rate remains high. However, the computational complexity of SR is too high for practical implementation, which is inherently associated with its iterative process. In this work, a deep neural network (DNN), which is trained with an incorporated loss function including sparse regularization terms, is proposed to reconstruct PWUS images from RF data with significantly reduced computational time. It is remarkable that, a self-supervised learning scheme, in which the RF data are utilized as both the inputs and the labels during the training process, is employed to overcome the lack of the "ideal" ultrasound images as the labels for DNN. In addition, it has been also verified that the trained network can be used on the RF data obtained with steered plane waves (PWs), and thus the image quality can be further improved with coherent compounding. Using simulation data, the proposed method has significantly shorter reconstruction time (∼10 ms) than the conventional SR method (∼1-5 mins), with comparable spatial resolution and 1.5-dB higher contrast-to-noise ratio (CNR). Besides, the proposed method with single PW can achieve higher CNR than DAS with 75 PWs in reconstruction of in-vivo images of human carotid arteries.

[1]  Ali Mousavi,et al.  Learning to invert: Signal recovery via Deep Convolutional Networks , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Jianwen Luo,et al.  High-Quality Reconstruction of Plane-Wave Imaging Using Generative Adversarial Network , 2018, 2018 IEEE International Ultrasonics Symposium (IUS).

[3]  A. Alexandrov,et al.  Carotid Artery Stenosis: Grayscale and Doppler Ultrasound Diagnosis—Society of Radiologists in Ultrasound Consensus Conference , 2003, Ultrasound quarterly.

[4]  Aaron O'Leary,et al.  PyWavelets: A Python package for wavelet analysis , 2019, J. Open Source Softw..

[5]  Jianwen Luo,et al.  Deep Learning for Ultrasound Localization Microscopy , 2020, IEEE Transactions on Medical Imaging.

[6]  Jean-Philippe Thiran,et al.  A deep learning approach to ultrasound image recovery , 2017, 2017 IEEE International Ultrasonics Symposium (IUS).

[7]  Orcun Goksel,et al.  Inverse Problem of Ultrasound Beamforming With Sparsity Constraints and Regularization , 2018, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[8]  Mathieu Couade,et al.  Noninvasive in vivo liver fibrosis evaluation using supersonic shear imaging: a clinical study on 113 hepatitis C virus patients. , 2011, Ultrasound in medicine & biology.

[9]  Jong Chul Ye,et al.  Adaptive and Compressive Beamforming Using Deep Learning for Medical Ultrasound , 2019, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[10]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[12]  Yonina C. Eldar,et al.  Deep Unfolded Robust PCA with Application to Clutter Suppression in Ultrasound , 2018, bioRxiv.

[13]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[14]  M. Fink,et al.  Functional ultrasound imaging of the brain: theory and basic principles , 2013, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[15]  H. Struijker‐Boudier,et al.  Expert consensus document on arterial stiffness: methodological issues and clinical applications. , 2006, European heart journal.

[16]  Giel Nijpels,et al.  Local stiffness of the carotid and femoral artery is associated with incident cardiovascular events and all-cause mortality: the Hoorn study. , 2014, Journal of the American College of Cardiology.

[17]  Hiroshi Kanai,et al.  Effect of subaperture beamforming on phase coherence imaging , 2014, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[18]  Shin Yoshizawa,et al.  Measurement of regional pulse-wave velocity using spatial compound imaging of the common carotid artery in vivo. , 2015, Ultrasonics.

[19]  Sverre Holm,et al.  Wiener beamforming and the coherence factor in ultrasound imaging , 2010, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[20]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[21]  Muyinatu A. Lediju Bell,et al.  Deep Learning to Obtain Simultaneous Image and Segmentation Outputs From a Single Input of Raw Ultrasound Channel Data , 2020, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[22]  Elisa E. Konofagou,et al.  Sparse Matrix Beamforming and Image Reconstruction for 2-D HIFU Monitoring Using Harmonic Motion Imaging for Focused Ultrasound (HMIFU) With In Vitro Validation , 2014, IEEE Transactions on Medical Imaging.

[23]  Gregg E. Trahey,et al.  Erratum: Sources of image degradation in fundamental and harmonic ultrasound imaging: A nonlinear, full-wave, simulation study (IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control) , 2011 .

[24]  Anikitos Garofalakis,et al.  Spatiotemporal matrix image formation for programmable ultrasound scanners , 2018, Physics in medicine and biology.

[25]  Suhyun Park,et al.  A fast minimum variance beamforming method using principal component analysis , 2014, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[26]  Giovanni Magenes,et al.  The Delay Multiply and Sum Beamforming Algorithm in Ultrasound B-Mode Medical Imaging , 2015, IEEE Transactions on Medical Imaging.

[27]  M. Fink,et al.  Coherent plane-wave compounding for very high frame rate ultrasonography and transient elastography , 2009, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[28]  Dongwoon Hyun,et al.  Beamforming and Speckle Reduction Using Neural Networks , 2019, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[29]  R. Virmani,et al.  Concept of vulnerable/unstable plaque. , 2010, Arteriosclerosis, thrombosis, and vascular biology.

[30]  D. Sackett,et al.  Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis. , 1991, The New England journal of medicine.

[31]  Charlie Demené,et al.  Spatiotemporal Clutter Filtering of Ultrafast Ultrasound Data Highly Increases Doppler and fUltrasound Sensitivity , 2015, IEEE Transactions on Medical Imaging.

[32]  M. Fink,et al.  Quantitative assessment of arterial wall biomechanical properties using shear wave imaging. , 2010, Ultrasound in medicine & biology.

[33]  E. Konofagou,et al.  Pulse wave imaging of the human carotid artery: an in vivo feasibility study , 2012, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[34]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[35]  Andrew F. Laine,et al.  On de-cluttering and de-noising ultrasound images with Time-domain Compressive Beamforming , 2016, 2016 IEEE International Ultrasonics Symposium (IUS).

[36]  Yonina C. Eldar,et al.  Sparsity-driven super-localization in clinical contrast-enhanced ultrasound , 2017, 2017 IEEE International Ultrasonics Symposium (IUS).

[37]  Damien Garcia,et al.  Noninvasive Vascular Elastography With Plane Strain Incompressibility Assumption Using Ultrafast Coherent Compound Plane Wave Imaging , 2015, IEEE Transactions on Medical Imaging.

[38]  T. Varghese,et al.  Methods for robust in vivo strain estimation in the carotid artery , 2012, Physics in medicine and biology.

[39]  Yonina C. Eldar,et al.  Deep Learning in Ultrasound Imaging , 2019, Proceedings of the IEEE.

[40]  Gordon S. Kino,et al.  A theory for the radiation pattern of a narrow‐strip acoustic transducer , 1980 .

[41]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[42]  Thomas Deffieux,et al.  Quantitative assessment of breast lesion viscoelasticity: initial clinical results using supersonic shear imaging. , 2008, Ultrasound in medicine & biology.

[43]  J. Jensen,et al.  Calculation of pressure fields from arbitrarily shaped, apodized, and excited ultrasound transducers , 1992, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[44]  Dongwoon Hyun,et al.  Reverberation Noise Suppression in the Aperture Domain Using 3D Fully Convolutional Neural Networks , 2018, 2018 IEEE International Ultrasonics Symposium (IUS).

[45]  Elisa E Konofagou,et al.  Validation of electromechanical wave imaging in a canine model during pacing and sinus rhythm. , 2016, Heart rhythm.

[46]  Enrico Grisan,et al.  Sparse Image Reconstruction for Contrast Enhanced Cardiac Ultrasound using Diverging Waves , 2019, 2019 IEEE International Ultrasonics Symposium (IUS).

[47]  Brett Byram,et al.  Deep Neural Networks for Ultrasound Beamforming , 2018, IEEE Transactions on Medical Imaging.

[48]  J. Jensen,et al.  Optimized Plane Wave Imaging for Fast and High-Quality Ultrasound Imaging. , 2016, IEEE transactions on ultrasonics, ferroelectrics, and frequency control.

[49]  Telmo Pereira,et al.  Aortic stiffness is an independent predictor of stroke in hypertensive patients. , 2013, Arquivos brasileiros de cardiologia.

[50]  Amir Asif,et al.  Multi-Focus Ultrasound Imaging Using Generative Adversarial Networks , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[51]  Denis Friboulet,et al.  High-Quality Plane Wave Compounding Using Convolutional Neural Networks , 2017, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[52]  Massimo Mischi,et al.  Adaptive Ultrasound Beamforming Using Deep Learning , 2019, IEEE Transactions on Medical Imaging.

[53]  Dong Liang,et al.  An easily-achieved time-domain beamformer for ultrafast ultrasound imaging based on compressive sensing , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[54]  Jie Tang,et al.  Prior image constrained compressed sensing (PICCS): a method to accurately reconstruct dynamic CT images from highly undersampled projection data sets. , 2008, Medical physics.

[55]  O. Bernard,et al.  Plane-Wave Imaging Challenge in Medical Ultrasound , 2016, 2016 IEEE International Ultrasonics Symposium (IUS).

[56]  Mickael Tanter,et al.  Ultrafast imaging in biomedical ultrasound , 2014 .

[57]  Lingyun Huang,et al.  Ultrasound-Based Carotid Elastography for Detection of Vulnerable Atherosclerotic Plaques Validated by Magnetic Resonance Imaging. , 2016, Ultrasound in medicine & biology.

[58]  Habib Zaidi,et al.  X-ray CT Metal Artifact Reduction Using Wavelet Domain $L_{0}$ Sparse Regularization , 2013, IEEE Transactions on Medical Imaging.

[59]  Jean-Philippe Thiran,et al.  A Sparse Reconstruction Framework for Fourier-Based Plane-Wave Imaging , 2016, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[60]  Bo Zhang,et al.  Time domain compressive beam forming of ultrasound signals. , 2015, The Journal of the Acoustical Society of America.

[61]  Jean-Philippe Thiran,et al.  Ultrafast Ultrasound Imaging as an Inverse Problem: Matrix-Free Sparse Image Reconstruction , 2018, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[62]  E. Konofagou,et al.  Noninvasive electromechanical wave imaging and conduction-relevant velocity estimation in vivo. , 2010, Ultrasonics.

[63]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[64]  Stephen P. Boyd,et al.  Robust minimum variance beamforming , 2005, IEEE Transactions on Signal Processing.

[65]  S. Laurent,et al.  Aortic Stiffness Is an Independent Predictor of Primary Coronary Events in Hypertensive Patients: A Longitudinal Study , 2002, Hypertension.

[66]  Alexander M. Bronstein,et al.  High frame-rate cardiac ultrasound imaging with deep learning , 2018, MICCAI.

[67]  Anne E. C. M. Saris,et al.  Ultrafast vascular strain compounding using plane wave transmission. , 2014, Journal of biomechanics.

[68]  M. Tanter,et al.  Ultrafast ultrasound localization microscopy for deep super-resolution vascular imaging , 2015, Nature.

[69]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[70]  Adrian Basarab,et al.  Beamforming Through Regularized Inverse Problems in Ultrasound Medical Imaging , 2015, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[71]  E. Konofagou,et al.  Pulse wave imaging using coherent compounding in a phantom and in vivo , 2017, Physics in medicine and biology.

[72]  J. V. van Engelshoven,et al.  Identifying vulnerable carotid plaques by noninvasive imaging , 2008, Neurology.

[73]  Dong Ni,et al.  Deep Learning in Medical Ultrasound Analysis: A Review , 2019, Engineering.

[74]  Jean-Philippe Thiran,et al.  Sparsity Averaging for Compressive Imaging , 2012, IEEE Signal Processing Letters.