Y-Net: Hybrid deep learning image reconstruction for photoacoustic tomography in vivo

Conventional reconstruction algorithms (e.g., delay-and-sum) used in photoacoustic imaging (PAI) provide a fast solution while many artifacts remain, especially for limited-view with ill-posed problem. In this paper, we propose a new convolutional neural network (CNN) framework Y-Net: a CNN architecture to reconstruct the initial PA pressure distribution by optimizing both raw data and beamformed images once. The network combines two encoders with one decoder path, which optimally utilizes more information from raw data and beamformed image. We compared our result with some ablation studies, and the results of the test set show better performance compared with conventional reconstruction algorithms and other deep learning method (U-Net). Both in-vitro and in-vivo experiments are used to validated our method, which still performs better than other existing methods. The proposed Y-Net architecture also has high potential in medical image reconstruction for other imaging modalities beyond PAI.

[1]  Vasilis Ntziachristos,et al.  Three-dimensional optoacoustic reconstruction using fast sparse representation. , 2017, Optics letters.

[2]  Yuanjin Zheng,et al.  Single laser pulse generates dual photoacoustic signals for differential contrast photoacoustic imaging , 2017, Scientific Reports.

[3]  Muyinatu A. Lediju Bell,et al.  Photoacoustic Source Detection and Reflection Artifact Removal Enabled by Deep Learning , 2018, IEEE Transactions on Medical Imaging.

[4]  Meng Zhou,et al.  Review of Low-Cost Photoacoustic Sensing and Imaging Based on Laser Diode and Light-Emitting Diode , 2018, Sensors.

[5]  M. Anastasio,et al.  Investigation of iterative image reconstruction in three-dimensional optoacoustic tomography , 2012, Physics in medicine and biology.

[6]  Muyinatu A. Lediju Bell,et al.  A machine learning approach to identifying point source locations in photoacoustic data , 2017, BiOS.

[7]  Da Xing,et al.  Three-dimensional photoacoustic imaging system in line confocal mode for breast cancer detection , 2010 .

[8]  Yiqiu Dong,et al.  An algorithm for total variation regularized photoacoustic imaging , 2015, Adv. Comput. Math..

[9]  Hao Chen,et al.  3D deeply supervised network for automated segmentation of volumetric medical images , 2017, Medical Image Anal..

[10]  Michael Unser,et al.  Deep Convolutional Neural Network for Inverse Problems in Imaging , 2016, IEEE Transactions on Image Processing.

[11]  Giovanni Magenes,et al.  The Delay Multiply and Sum Beamforming Algorithm in Ultrasound B-Mode Medical Imaging , 2015, IEEE Transactions on Medical Imaging.

[12]  Lihong V. Wang,et al.  A practical guide to photoacoustic tomography in the life sciences , 2016, Nature Methods.

[13]  Mark A. Anastasio,et al.  Full-Wave Iterative Image Reconstruction in Photoacoustic Tomography With Acoustically Inhomogeneous Media , 2013, IEEE Transactions on Medical Imaging.

[14]  Lihong V. Wang,et al.  Tutorial on Photoacoustic Microscopy and Computed Tomography , 2008, IEEE Journal of Selected Topics in Quantum Electronics.

[15]  Lihong V. Wang,et al.  Single-breath-hold photoacoustic computed tomography of the breast , 2018, Nature Communications.

[16]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[17]  Yuanjin Zheng,et al.  Single-Wavelength Blood Oxygen Saturation Sensing With Combined Optical Absorption and Scattering , 2016, IEEE Sensors Journal.

[18]  Lihong V. Wang,et al.  Photoacoustic imaging in biomedicine , 2006 .

[19]  Andreas Hauptmann,et al.  Model-Based Learning for Accelerated, Limited-View 3-D Photoacoustic Tomography , 2017, IEEE Transactions on Medical Imaging.

[20]  Rita Strack,et al.  Imaging: AI transforms image reconstruction , 2018, Nature Methods.

[21]  Moein Mozaffarzadeh,et al.  Photoacoustic image formation based on sparse regularization of minimum variance beamformer. , 2018, Biomedical optics express.

[22]  S. Camou,et al.  Detection of aqueous glucose based on a cavity size- and optical-wavelength-independent continuous-wave photoacoustic technique. , 2012, Analytical chemistry.

[23]  Lihong V Wang,et al.  Universal back-projection algorithm for photoacoustic computed tomography , 2005, SPIE BiOS.

[24]  Meng Zhou,et al.  Photoacoustic Classification of Tumor Model Morphology Based on Support Vector Machine: A Simulation and Phantom Study , 2019, IEEE Journal of Selected Topics in Quantum Electronics.

[25]  Stephan Antholzer,et al.  DALnet: High-resolution photoacoustic projection imaging using deep learning , 2018 .

[26]  Markus Haltmeier,et al.  A U-Nets Cascade for Sparse View Computed Tomography , 2018, MLMIR@MICCAI.

[27]  Shenghua Gao,et al.  Hybrid Neural Network for Photoacoustic Imaging Reconstruction , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[28]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[29]  Meng Zhou,et al.  Hybrid multi-wavelength nonlinear photoacoustic sensing and imaging. , 2018, Optics letters.

[30]  Zhuowen Tu,et al.  Deeply-Supervised Nets , 2014, AISTATS.

[31]  Jin U. Kang,et al.  Enabling fast and high quality LED photoacoustic imaging: a recurrent neural networks based approach. , 2018, Biomedical optics express.

[32]  Mathews Jacob,et al.  MoDL: Model-Based Deep Learning Architecture for Inverse Problems , 2017, IEEE Transactions on Medical Imaging.

[33]  Stephan Antholzer,et al.  NETT: solving inverse problems with deep neural networks , 2018, Inverse Problems.

[34]  Bin Dong,et al.  A Review on Deep Learning in Medical Image Reconstruction , 2019, Journal of the Operations Research Society of China.

[35]  Lihong V. Wang,et al.  Photoacoustic Tomography: In Vivo Imaging from Organelles to Organs , 2012, Science.

[36]  Janek Gröhl,et al.  Reconstruction of initial pressure from limited view photoacoustic images using deep learning , 2018, BiOS.

[37]  Andreas K. Maier,et al.  A Deep Learning Architecture for Limited-Angle Computed Tomography Reconstruction , 2017, Bildverarbeitung für die Medizin.

[38]  Chen Zhang,et al.  Total variation based gradient descent algorithm for sparse-view photoacoustic image reconstruction. , 2012, Ultrasonics.

[39]  Ali Mahloojifar,et al.  Double-Stage Delay Multiply and Sum Beamforming Algorithm: Application to Linear-Array Photoacoustic Imaging , 2018, IEEE Transactions on Biomedical Engineering.

[40]  Jianwen Luo,et al.  End-to-end deep neural network for optical inversion in quantitative photoacoustic imaging. , 2018, Optics letters.

[41]  Max A. Viergever,et al.  Ridge-based vessel segmentation in color images of the retina , 2004, IEEE Transactions on Medical Imaging.

[42]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[43]  Jin U. Kang,et al.  Towards a Fast and Safe LED-Based Photoacoustic Imaging Using Deep Convolutional Neural Network , 2018, MICCAI.

[44]  Jeffrey A. Fessler,et al.  Image Reconstruction is a New Frontier of Machine Learning , 2018, IEEE Transactions on Medical Imaging.

[45]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[46]  Manojit Pramanik,et al.  Fractional Regularization to Improve Photoacoustic Tomographic Image Reconstruction , 2019, IEEE Transactions on Medical Imaging.

[47]  Werner Mäntele,et al.  In vivo noninvasive monitoring of glucose concentration in human epidermis by mid-infrared pulsed photoacoustic spectroscopy. , 2013, Analytical chemistry.

[48]  Meng Zhou,et al.  Dual-Contrast Nonlinear Photoacoustic Sensing and Imaging Based on Single High-Repetition- Rate Pulsed Laser , 2019, IEEE Sensors Journal.

[49]  Lihong V. Wang,et al.  Grueneisen relaxation photoacoustic microscopy. , 2014, Physical review letters.

[50]  Yong Zhou,et al.  Tutorial on photoacoustic tomography , 2016, Journal of biomedical optics.

[51]  Stephan Antholzer,et al.  Photoacoustic image reconstruction via deep learning , 2018, BiOS.

[52]  Parsa Omidi,et al.  A Novel Dictionary-Based Image Reconstruction for Photoacoustic Computed Tomography , 2018, Applied Sciences.

[53]  Yoeri E Boink,et al.  A Partially-Learned Algorithm for Joint Photo-acoustic Reconstruction and Segmentation , 2019, IEEE Transactions on Medical Imaging.

[54]  Paul C. Beard,et al.  Approximate k-space models and Deep Learning for fast photoacoustic reconstruction , 2018, MLMIR@MICCAI.

[55]  B T Cox,et al.  k-Wave: MATLAB toolbox for the simulation and reconstruction of photoacoustic wave fields. , 2010, Journal of biomedical optics.

[56]  Stephan Antholzer,et al.  NETT regularization for compressed sensing photoacoustic tomography , 2019, BiOS.

[57]  Hao Zhang,et al.  Imaging of hemoglobin oxygen saturation variations in single vessels in vivo using photoacoustic microscopy , 2007 .