Neural network-based image reconstruction in swept-source optical coherence tomography using undersampled spectral data

Optical coherence tomography (OCT) is a widely used non-invasive biomedical imaging modality that can rapidly provide volumetric images of samples. Here, we present a deep learning-based image reconstruction framework that can generate swept-source OCT (SS-OCT) images using undersampled spectral data, without any spatial aliasing artifacts. This neural network-based image reconstruction does not require any hardware changes to the optical setup and can be easily integrated with existing swept-source or spectral-domain OCT systems to reduce the amount of raw spectral data to be acquired. To show the efficacy of this framework, we trained and blindly tested a deep neural network using mouse embryo samples imaged by an SS-OCT system. Using 2-fold undersampled spectral data (i.e., 640 spectral points per A-line), the trained neural network can blindly reconstruct 512 A-lines in 0.59 ms using multiple graphics-processing units (GPUs), removing spatial aliasing artifacts due to spectral undersampling, also presenting a very good match to the images of the same samples, reconstructed using the full spectral OCT data (i.e., 1280 spectral points per A-line). We also successfully demonstrate that this framework can be further extended to process 3× undersampled spectral data per A-line, with some performance degradation in the reconstructed image quality compared to 2× spectral undersampling. Furthermore, an A-line-optimized undersampling method is presented by jointly optimizing the spectral sampling locations and the corresponding image reconstruction network, which improved the overall imaging performance using less spectral data points per A-line compared to 2× or 3× spectral undersampling results. This deep learning-enabled image reconstruction approach can be broadly used in various forms of spectral-domain OCT systems, helping to increase their imaging speed without sacrificing image resolution and signal-to-noise ratio.

[1]  S. Dwivedi,et al.  Obesity May Be Bad: Compressed Convolutional Networks for Biomedical Image Segmentation , 2020 .

[2]  Giovanni Volpe,et al.  Digital video microscopy enhanced by deep learning , 2018, Optica.

[3]  A. Ozcan,et al.  On the use of deep learning for computational imaging , 2019, Optica.

[4]  J. Fujimoto,et al.  Three-dimensional endomicroscopy using optical coherence tomography , 2007 .

[5]  Wolfgang Drexler,et al.  Optical coherence tomography: Technology and applications , 2013, 2013 Conference on Lasers & Electro-Optics Europe & International Quantum Electronics Conference CLEO EUROPE/IQEC.

[6]  J. Fujimoto,et al.  Optical coherence tomography using a frequency-tunable optical source. , 1997, Optics letters.

[7]  Aydogan Ozcan,et al.  Deep-Learning-Based Image Reconstruction and Enhancement in Optical Microscopy , 2020, Proceedings of the IEEE.

[8]  J. Fujimoto,et al.  Optical Coherence Tomography , 1991 .

[9]  Chen Wu,et al.  Applicability, usability, and limitations of murine embryonic imaging with optical coherence tomography and optical projection tomography. , 2016, Biomedical optics express.

[10]  Van Lam,et al.  Automatic phase aberration compensation for digital holographic microscopy based on deep learning background detection. , 2017, Optics express.

[11]  Christophe Zimmer,et al.  Deep learning massively accelerates super-resolution localization microscopy , 2018, Nature Biotechnology.

[12]  Thomas Klein,et al.  High-speed OCT light sources and systems [Invited]. , 2017, Biomedical optics express.

[13]  Aydogan Ozcan,et al.  Single-shot autofocusing of microscopy images using deep learning , 2020, ArXiv.

[14]  Yibo Zhang,et al.  Deep learning‐based color holographic microscopy , 2019, Journal of biophotonics.

[15]  Tomer Michaeli,et al.  Multicolor localization microscopy and point-spread-function engineering by deep learning. , 2019, Optics express.

[16]  R. Leitgeb,et al.  Ultrahigh-speed non-invasive widefield angiography. , 2012, Journal of biomedical optics.

[17]  Kenneth K Y Wong,et al.  28 MHz swept source at 1.0 μm for ultrafast quantitative phase imaging. , 2015, Biomedical optics express.

[18]  J. Fujimoto,et al.  Fourier Domain Mode Locking (FDML): A new laser operating regime and applications for optical coherence tomography. , 2006, Optics express.

[19]  A. Ozcan,et al.  Deep learning enables cross-modality super-resolution in fluorescence microscopy , 2018, Nature Methods.

[20]  R. Leitgeb,et al.  Twenty-five years of optical coherence tomography: the paradigm shift in sensitivity and speed provided by Fourier domain OCT [Invited]. , 2017, Biomedical optics express.

[21]  J. Fujimoto,et al.  Ultrahigh speed spectral / Fourier domain OCT ophthalmic imaging at 70,000 to 312,500 axial scans per second. , 2008, Optics express.

[22]  Henry Pinkard,et al.  Deep learning for single-shot autofocus microscopy , 2019, Optica.

[23]  Martin F. Kraus,et al.  Ultrahigh speed endoscopic optical coherence tomography using micromotor imaging catheter and VCSEL technology , 2013, Photonics West - Biomedical Optics.

[24]  Changhuei Yang,et al.  Sensitivity advantage of swept source and Fourier domain optical coherence tomography. , 2003, Optics express.

[25]  Alexander W. Schill,et al.  Phase-sensitive optical coherence elastography at 1.5 million A-Lines per second. , 2015, Optics letters.

[26]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[27]  John A. Evans,et al.  Comprehensive volumetric optical microscopy in vivo , 2006, Nature Medicine.

[28]  B. Bouma,et al.  Improved signal-to-noise ratio in spectral-domain compared with time-domain optical coherence tomography. , 2003, Optics letters.

[29]  Yibo Zhang,et al.  Deep Learning Microscopy , 2017, ArXiv.

[30]  Aydogan Ozcan,et al.  Resolution enhancement in scanning electron microscopy using deep learning , 2019, Scientific Reports.

[31]  A. Ozcan,et al.  Recurrent neural network-based volumetric fluorescence microscopy , 2020, Light, science & applications.

[32]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[33]  B. Vakoc,et al.  >400 kHz repetition rate wavelength-swept laser and application to high-speed optical frequency domain imaging. , 2010, Optics letters.

[34]  S. Yun,et al.  Ultrahigh-speed optical frequency domain imaging and application to laser ablation monitoring , 2006 .

[35]  Tomer Michaeli,et al.  Deep-STORM: super-resolution single-molecule microscopy by deep learning , 2018, 1801.09631.

[36]  Lei Tian,et al.  Deep learning approach for Fourier ptychography microscopy. , 2018, Optics express.

[37]  Aydogan Ozcan,et al.  Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning , 2019, Nature Methods.

[38]  Kevin de Haan,et al.  Deep learning-based holographic polarization microscopy , 2020, ACS photonics.

[39]  Mohammad Shorif Uddin,et al.  Image Quality Assessment through FSIM, SSIM, MSE and PSNR—A Comparative Study , 2019, Journal of Computer and Communications.

[40]  J. Fujimoto,et al.  Buffered Fourier domain mode locking: Unidirectional swept laser sources for optical coherence tomography imaging at 370,000 lines/s. , 2006, Optics letters.

[41]  Yibo Zhang,et al.  Phase recovery and holographic image reconstruction using deep learning in neural networks , 2017, Light: Science & Applications.

[42]  Yibo Zhang,et al.  Extended depth-of-field in holographic image reconstruction using deep learning based auto-focusing and phase-recovery , 2018, Optica.

[43]  Lei Tian,et al.  Deep learning approach to Fourier ptychographic microscopy , 2018, 1805.00334.

[44]  Wolfgang Wieser,et al.  High definition live 3D-OCT in vivo: design and evaluation of a 4D OCT engine with 1 GVoxel/s. , 2014, Biomedical optics express.

[45]  G. Ripandelli,et al.  Optical coherence tomography. , 1998, Seminars in ophthalmology.

[46]  A. Fercher,et al.  Measurement of intraocular distances by backscattering spectral interferometry , 1995 .

[47]  Martin F. Kraus,et al.  Total retinal blood flow measurement with ultrahigh speed swept source/Fourier domain OCT , 2011, Biomedical optics express.

[48]  Thomas J. Naughton,et al.  Performance of autofocus capability of deep convolutional neural networks in digital holographic microscopy , 2016 .