Deep-Learning Image Reconstruction for Real-Time Photoacoustic System

Recent advances in photoacoustic (PA) imaging have enabled detailed images of microvascular structure and quantitative measurement of blood oxygenation or perfusion. Standard reconstruction methods for PA imaging are based on solving an inverse problem using appropriate signal and system models. For handheld scanners, however, the ill-posed conditions of limited detection view and bandwidth yield low image contrast and severe structure loss in most instances. In this paper, we propose a practical reconstruction method based on a deep convolutional neural network (CNN) to overcome those problems. It is designed for real-time clinical applications and trained by large-scale synthetic data mimicking typical microvessel networks. Experimental results using synthetic and real datasets confirm that the deep-learning approach provides superior reconstructions compared to conventional methods.

[1]  Yuan Xu,et al.  Exact frequency-domain reconstruction for thermoacoustic tomography. I. Planar geometry , 2002, IEEE Transactions on Medical Imaging.

[2]  Minghua Xu,et al.  Time-domain reconstruction for thermoacoustic tomography in a spherical geometry , 2002, IEEE Transactions on Medical Imaging.

[3]  Max A. Viergever,et al.  Ridge-based vessel segmentation in color images of the retina , 2004, IEEE Transactions on Medical Imaging.

[4]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[5]  Lihong V. Wang,et al.  Photoacoustic imaging in biomedicine , 2006 .

[6]  G. Farhat,et al.  Diagnostic ultrasound Imaging : Inside out , 2004 .

[7]  Lihong V. Wang,et al.  Biomedical Optics: Principles and Imaging , 2007 .

[8]  Suhyun Park,et al.  Adaptive beamforming for photoacoustic imaging. , 2008, Optics letters.

[9]  David A Boas,et al.  Monte Carlo simulation of photon migration in 3D turbid media accelerated by graphics processing units. , 2009, Optics express.

[10]  Vasilis Ntziachristos,et al.  Fast Semi-Analytical Model-Based Acoustic Inversion for Quantitative Optoacoustic Tomography , 2010, IEEE Transactions on Medical Imaging.

[11]  Yann LeCun,et al.  Learning Fast Approximations of Sparse Coding , 2010, ICML.

[12]  Lihong V Wang,et al.  Compressed sensing in photoacoustic tomography in vivo. , 2010, Journal of biomedical optics.

[13]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[14]  Markus Haltmeier,et al.  Inversion of circular means and the wave equation on convex planar domains , 2012, Comput. Math. Appl..

[15]  Lihong V. Wang,et al.  Small-Animal Whole-Body Photoacoustic Tomography: A Review , 2014, IEEE Transactions on Biomedical Engineering.

[16]  Lihong V. Wang,et al.  Photoacoustic tomography: principles and advances. , 2016, Electromagnetic waves.

[17]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[18]  H. Hasegawa,et al.  Effect of element directivity on adaptive beamforming applied to high-frame-rate ultrasound , 2015, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[19]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[20]  H. Hasegawa,et al.  Erratum: Effect of element directivity on adaptive beamforming applied to high-frame-rate ultrasound. , 2015, IEEE transactions on ultrasonics, ferroelectrics, and frequency control.

[21]  Giovanni Magenes,et al.  The Delay Multiply and Sum Beamforming Algorithm in Ultrasound B-Mode Medical Imaging , 2015, IEEE Transactions on Medical Imaging.

[22]  Minghua Xu,et al.  Universal Back-Projection Algorithm for Photoacoustic Tomography , 2017 .

[23]  Yue Lu,et al.  A Tale of Two Bases: Local-Nonlocal Regularization on Image Patches with Convolution Framelets , 2016, SIAM J. Imaging Sci..

[24]  Michael Unser,et al.  Convolutional Neural Networks for Inverse Problems in Imaging: A Review , 2017, IEEE Signal Processing Magazine.

[25]  June-Goo Lee,et al.  Deep Learning in Medical Imaging: General Overview , 2017, Korean journal of radiology.

[26]  Leslie Pack Kaelbling,et al.  Generalization in Deep Learning , 2017, ArXiv.

[27]  Yi Shen,et al.  Compressed sensing in synthetic aperture photoacoustic tomography based on a linear-array ultrasound transducer , 2017 .

[28]  Michael Unser,et al.  Deep Convolutional Neural Network for Inverse Problems in Imaging , 2016, IEEE Transactions on Image Processing.

[29]  Jianwen Luo,et al.  End-to-end deep neural network for optical inversion in quantitative photoacoustic imaging. , 2018, Optics letters.

[30]  Jesse V. Jokerst,et al.  The double-stage delay-multiply-and-sum image reconstruction method improves imaging quality in a LED-based photoacoustic array scanner , 2018, Photoacoustics.

[31]  Bo Wang,et al.  Finite-element reconstruction of 2D circular scanning photoacoustic tomography with detectors in far-field condition. , 2018, Applied optics.

[32]  Lena Maier-Hein,et al.  Signed Real-Time Delay Multiply and Sum Beamforming for Multispectral Photoacoustic Imaging , 2018, J. Imaging.

[33]  S. Han Review of Photoacoustic Imaging for Imaging-Guided Spinal Surgery , 2018, Neurospine.

[34]  Paul C. Beard,et al.  Approximate k-space models and Deep Learning for fast photoacoustic reconstruction , 2018, MLMIR@MICCAI.

[35]  Jong Chul Ye,et al.  Deep Convolutional Framelets: A General Deep Learning Framework for Inverse Problems , 2017, SIAM J. Imaging Sci..

[36]  Heather K. Hunt,et al.  Hand-held optoacoustic imaging: A review , 2018, Photoacoustics.

[37]  Muyinatu A. Lediju Bell,et al.  Photoacoustic Source Detection and Reflection Artifact Removal Enabled by Deep Learning , 2018, IEEE Transactions on Medical Imaging.

[38]  Bruce R. Rosen,et al.  Image reconstruction by domain-transform manifold learning , 2017, Nature.

[39]  Jiaying Xiao,et al.  Back-projection algorithm in generalized form for circular-scanning-based photoacoustic tomography with improved tangential resolution. , 2019, Quantitative imaging in medicine and surgery.

[40]  Phaneendra K. Yalavarthy,et al.  PA-Fuse: deep supervised approach for the fusion of photoacoustic images with distinct reconstruction characteristics. , 2019, Biomedical optics express.

[41]  Sanjiv S. Gambhir,et al.  Photoacoustic clinical imaging , 2019, Photoacoustics.

[42]  M. O’Donnell,et al.  Real-time swept-beam compact photoacoustic/ultrasound imaging system (Conference Presentation) , 2019, Photons Plus Ultrasound: Imaging and Sensing 2019.

[43]  Chulhong Kim,et al.  Real-time delay-multiply-and-sum beamforming with coherence factor for in vivo clinical photoacoustic imaging of humans , 2019, Photoacoustics.

[44]  Geng-Shi Jeng,et al.  Real-time spectroscopic photoacoustic/ultrasound (PAUS) scanning with simultaneous fluence compensation and motion correction for quantitative molecular imaging , 2019, bioRxiv.

[45]  Paul Babyn,et al.  Generative Adversarial Network in Medical Imaging: A Review , 2018, Medical Image Anal..

[46]  Stephan Antholzer,et al.  Deep learning for photoacoustic tomography from sparse data , 2017, Inverse problems in science and engineering.

[47]  Xosé Luís Deán-Ben,et al.  Deep learning optoacoustic tomography with sparse data , 2019, Nat. Mach. Intell..

[48]  Mathews Jacob,et al.  MoDL: Model-Based Deep Learning Architecture for Inverse Problems , 2017, IEEE Transactions on Medical Imaging.

[49]  Vasilis Ntziachristos,et al.  A review of clinical photoacoustic imaging: Current and future trends , 2019, Photoacoustics.

[50]  Jong Chul Ye,et al.  Adaptive and Compressive Beamforming Using Deep Learning for Medical Ultrasound , 2019, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[51]  Massimo Mischi,et al.  Adaptive Ultrasound Beamforming Using Deep Learning , 2019, IEEE Transactions on Medical Imaging.

[52]  Yoeri E Boink,et al.  A Partially-Learned Algorithm for Joint Photo-acoustic Reconstruction and Segmentation , 2019, IEEE Transactions on Medical Imaging.