Speckle Noise Reduction for OCT Images Based on Image Style Transfer and Conditional GAN

Raw optical coherence tomography (OCT) images typically are of low quality because speckle noise blurs retinal structures, severely compromising visual quality and degrading performances of subsequent image analysis tasks. In our previous study (Ma et al., 2018), we have developed a Conditional Generative Adversarial Network (cGAN) for speckle noise removal in OCT images collected by several commercial OCT scanners, which we collectively refer to as scanner T. In this paper, we improve the cGAN model and apply it to our in-house OCT scanner (scanner B) for speckle noise suppression. The proposed model consists of two steps: 1) We train a Cycle-Consistent GAN (CycleGAN) to learn style transfer between two OCT image datasets collected by different scanners. The purpose of the CycleGAN is to leverage the ground truth dataset created in our previous study. 2) We train a mini-cGAN model based on the PatchGAN mechanism with the ground truth dataset to suppress speckle noise in OCT images. After training, we first apply the CycleGAN model to convert raw images collected by scanner B to match the style of the images from scanner T, and subsequently use the mini-cGAN model to suppress speckle noise in the style transferred images. We evaluate the proposed method on a dataset collected by scanner B. Experimental results show that the improved model outperforms our previous method and other state-of-the-art models in speckle noise removal, retinal structure preservation and contrast enhancement.

[1]  Shenghua Gao,et al.  Noise Adaptation Generative Adversarial Network for Medical Image Analysis , 2020, IEEE Transactions on Medical Imaging.

[2]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[4]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[5]  Lian Duan,et al.  Single-shot speckle noise reduction by interleaved optical coherence tomography , 2014, Journal of biomedical optics.

[6]  Xinjian Chen,et al.  Enhanced low-rank + sparsity decomposition for speckle reduction in optical coherence tomography , 2016, Journal of biomedical optics.

[7]  Wolfgang Heidrich,et al.  Statistical model for OCT image denoising. , 2017, Biomedical optics express.

[8]  Ivan W. Selesnick,et al.  Three Dimensional Data-Driven Multi Scale Atomic Representation of Optical Coherence Tomography , 2015, IEEE Transactions on Medical Imaging.

[9]  Jiebo Luo,et al.  Adversarial Sparse-View CBCT Artifact Reduction , 2018, MICCAI.

[10]  Guangming Shi,et al.  Denoising Prior Driven Deep Neural Network for Image Restoration , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Shutao Li,et al.  Segmentation Based Sparse Reconstruction of Optical Coherence Tomography Images , 2017, IEEE Transactions on Medical Imaging.

[12]  Xinjian Chen,et al.  Dual-beam angular compounding for speckle reduction in optical coherence tomography , 2017, BiOS.

[13]  Jichai Jeong,et al.  Effective speckle noise suppression in optical coherence tomography images using nonlocal means denoising filter with double Gaussian anisotropic kernels , 2015 .

[14]  Verónica Vilaplana,et al.  Brain MRI super-resolution using 3D generative adversarial networks , 2018, ArXiv.

[15]  Raja Giryes,et al.  Class-Aware Fully Convolutional Gaussian and Poisson Denoising , 2018, IEEE Transactions on Image Processing.

[16]  Ali Hojjatoleslami,et al.  OCT skin image enhancement through attenuation compensation. , 2012, Applied optics.

[17]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[18]  Guang Li,et al.  CT Super-Resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE) , 2018, IEEE Transactions on Medical Imaging.

[19]  Dwarikanath Mahapatra,et al.  Deformable medical image registration using generative adversarial networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[20]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[21]  Xinjian Chen,et al.  Spiking cortical model–based nonlocal means method for speckle reduction in optical coherence tomography images , 2014, Journal of biomedical optics.

[22]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[24]  Jelmer M. Wolterink,et al.  Deep MR to CT Synthesis Using Unpaired Data , 2017, SASHIMI@MICCAI.

[25]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[27]  A. Curatolo,et al.  Speckle reduction in optical coherence tomography by strain compounding. , 2010, Optics letters.

[28]  G. Ripandelli,et al.  Optical coherence tomography. , 1998, Seminars in ophthalmology.

[29]  Kostadinka Bizheva,et al.  Stochastic speckle noise compensation in optical coherence tomography using non-stationary spline-based speckle noise modelling. , 2013, Biomedical optics express.

[30]  Sheng Xu,et al.  Adversarial Image Registration with Application for MR and TRUS Image Fusion , 2018, MLMI@MICCAI.

[31]  Ender Konukoglu,et al.  Generative Adversarial Networks for MR-CT Deformable Image Registration , 2018, ArXiv.

[32]  Alexander Wong,et al.  General Bayesian estimation for speckle noise reduction in optical coherence tomography retinal imagery. , 2010, Optics express.

[33]  Shutao Li,et al.  Sparsity based denoising of spectral domain optical coherence tomography images , 2012, Biomedical optics express.

[34]  Xinjian Chen,et al.  CPFNet: Context Pyramid Fusion Network for Medical Image Segmentation , 2020, IEEE Transactions on Medical Imaging.

[35]  Jorge Herbert de Lira,et al.  Two-Dimensional Signal and Image Processing , 1989 .

[36]  Benoit M. Dawant,et al.  Conditional Generative Adversarial Networks for Metal Artifact Reduction in CT Images of the Ear , 2018, MICCAI.

[37]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[38]  Kostadinka Bizheva,et al.  Interval type-II fuzzy anisotropic diffusion algorithm for speckle noise reduction in optical coherence tomography images. , 2009, Optics express.

[39]  Yanjun Li,et al.  When sparsity meets low-rankness: Transform learning with non-local low-rank constraint for image restoration , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  Alexander A. Sawchuk,et al.  Adaptive Noise Smoothing Filter for Images with Signal-Dependent Noise , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  J. Fujimoto,et al.  In vivo retinal imaging by optical coherence tomography. , 1993, Optics letters.

[42]  Xin Yuan,et al.  Noise adaptive wavelet thresholding for speckle noise removal in optical coherence tomography. , 2017, Biomedical optics express.

[43]  Bo Chong,et al.  Speckle reduction in optical coherence tomography images of human finger skin by wavelet modified BM3D filter , 2013 .

[44]  E. Nezry,et al.  Adaptive speckle filters and scene heterogeneity , 1990 .

[45]  Dacheng Tao,et al.  Speckle Reduction in 3D Optical Coherence Tomography of Retina by A-Scan Reconstruction , 2016, IEEE Trans. Medical Imaging.

[46]  Jian Sun,et al.  BM3D-Net: A Convolutional Neural Network for Transform-Domain Collaborative Filtering , 2018, IEEE Signal Processing Letters.

[47]  A. Fercher,et al.  In vivo human retinal imaging by Fourier domain optical coherence tomography. , 2002, Journal of biomedical optics.

[48]  Dwarikanath Mahapatra,et al.  Joint Registration And Segmentation Of Xray Images Using Generative Adversarial Networks , 2018, MLMI@MICCAI.

[49]  Jong-Sen Lee,et al.  Speckle analysis and smoothing of synthetic aperture radar images , 1981 .

[50]  Zhiwen Cao,et al.  PDE-Based Non-Linear Anisotropic Diffusion Techniques for Medical Image Denoising , 2012, 2012 Spring Congress on Engineering and Technology.

[51]  Bruce J. Tromberg,et al.  Three-dimensional speckle suppression in optical coherence tomography based on the curvelet transform , 2010, Optics express.

[52]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.