Computational optical tomography using 3-D deep convolutional neural networks

Abstract. Deep convolutional neural networks (DCNNs) offer a promising performance for many image processing areas, such as super-resolution, deconvolution, image classification, denoising, and segmentation, with outstanding results. Here, we develop for the first time, to our knowledge, a method to perform 3-D computational optical tomography using 3-D DCNN. A simulated 3-D phantom dataset was first constructed and converted to a dataset of phase objects imaged on a spatial light modulator. For each phase image in the dataset, the corresponding diffracted intensity image was experimentally recorded on a CCD. We then experimentally demonstrate the ability of the developed 3-D DCNN algorithm to solve the inverse problem by reconstructing the 3-D index of refraction distributions of test phantoms from the dataset from their corresponding diffraction patterns.

[1]  Geoffrey E. Hinton,et al.  On the importance of initialization and momentum in deep learning , 2013, ICML.

[2]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[3]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[4]  Christophe Moser,et al.  Versatile spectral modulation of a broadband source for digital holographic microscopy. , 2016, Optics express.

[5]  Yibo Zhang,et al.  Deep Learning Microscopy , 2017, ArXiv.

[6]  Ce Liu,et al.  Deep Convolutional Neural Network for Image Deconvolution , 2014, NIPS.

[7]  Justin Lee,et al.  Lensless computational imaging through deep learning , 2017, ArXiv.

[8]  Yu Wu,et al.  Simple and flexible phase compensation for digital holographic microscopy with electrically tunable lens. , 2017, Applied optics.

[9]  YongKeun Park,et al.  High-resolution three-dimensional imaging of red blood cells parasitized by Plasmodium falciparum and in situ hemozoin crystals using optical diffraction tomography , 2013, Journal of biomedical optics.

[10]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  I. Iglesias.  Pyramid phase microscopy. , 2011, Optics letters.

[12]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[14]  Pavan K. Turaga,et al.  ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[16]  Michael Unser,et al.  Deep Convolutional Neural Network for Inverse Problems in Imaging , 2016, IEEE Transactions on Image Processing.

[17]  Zhaoyang Wang,et al.  Three-dimensional facial digitization using advanced digital image correlation. , 2018, Applied optics.

[18]  Osamu Matoba,et al.  Three-dimensional imaging of distribution of refractive index by parallel phase-shifting digital holography using Abel inversion. , 2017, Optics express.

[19]  Martin Vetterli,et al.  Adaptive wavelet thresholding for image denoising and compression , 2000, IEEE Trans. Image Process..

[20]  Tony F. Chan,et al.  Total variation blind deconvolution , 1998, IEEE Trans. Image Process..

[21]  Bing Zhao,et al.  3D shape, deformation, and vibration measurements using infrared Kinect sensors and digital image correlation. , 2017, Applied optics.

[22]  Gaël Varoquaux,et al.  Mayavi: 3D Visualization of Scientific Data , 2010, Computing in Science & Engineering.

[23]  Yann LeCun,et al.  Learning Fast Approximations of Sparse Coding , 2010, ICML.

[24]  M. Unser,et al.  Interpolation revisited [medical images application] , 2000, IEEE Transactions on Medical Imaging.

[25]  A. Chambolle.  Practical, Unified, Motion and Missing Data Treatment in Degraded Video , 2004, Journal of Mathematical Imaging and Vision.

[26]  Michael Unser,et al.  Convolutional Neural Networks for Inverse Problems in Imaging: A Review , 2017, IEEE Signal Processing Magazine.

[27]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[28]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[29]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[30]  Shouyu Wang,et al.  Real-time quantitative phase imaging based on transport of intensity equation with dual simultaneously recorded field of view. , 2016, Optics letters.

[31]  Xiaoou Tang,et al.  Compression Artifacts Reduction by a Deep Convolutional Network , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[32]  J. Vane,et al.  Optical Projection Tomography as a Tool for 3D Microscopy and Gene Expression Studies , 2002 .

[33]  Van Lam,et al.  Automatic phase aberration compensation for digital holographic microscopy based on deep learning background detection. , 2017, Optics express.

[34]  J. Tanida,et al.  Learning-based imaging through scattering media. , 2016, Optics express.

[35]  A. Devaney.  Inverse-scattering theory within the Rytov approximation. , 1981, Optics letters.

[36]  Bradley L. Preece,et al.  A noise model for the design of a compressive sensing imaging system , 2015, Defense + Security Symposium.

[37]  Svoboda Pavel,et al.  CNN for license plate motion deblurring , 2016 .

[38]  Aggelos K. Katsaggelos,et al.  Using Deep Neural Networks for Inverse Problems in Imaging: Beyond Analytical Methods , 2018, IEEE Signal Processing Magazine.

[39]  Jong Chul Ye,et al.  k‐t FOCUSS: A general compressed sensing framework for high resolution dynamic MRI , 2009, Magnetic resonance in medicine.

[40]  Ignacio Iglesias,et al.  Tomographic imaging of transparent biological samples using the pyramid phase microscope. , 2016, Biomedical optics express.

[41]  Bin Sheng,et al.  Deep Colorization , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[42]  C. Fang-Yen,et al.  Optical diffraction tomography for high resolution live cell imaging. , 2009, Optics express.

[43]  G. Nehmetallah,et al.  Digital tomographic compressive holographic reconstruction of three-dimensional objects in transmissive and reflective geometries. , 2013, Applied optics.

[44]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Andrew L. Maas.  Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[46]  Julian Togelius,et al.  Deep Learning for Video Game Playing , 2017, IEEE Transactions on Games.

[47]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..