AdaIN-Switchable CycleGAN for Efficient Unsupervised Low-Dose CT Denoising

Recently, deep learning approaches have been extensively studied for low-dose CT denoising thanks to its superior performance despite the fast computational time. In particular, cycleGAN has been demonstrated as a powerful unsupervised learning scheme to improve the low-dose CT image quality without requiring matched high-dose reference data. Unfortunately, one of the main limitations of the cycleGAN approach is that it requires two deep neural network generators at the training phase, although only one of them is used at the inference phase. The secondary auxiliary generator is needed to enforce the cycle-consistency, but the additional memory requirement and increases of the learnable parameters are the main huddles for cycleGAN training. To address this issue, here we propose a novel cycleGAN architecture using a single switchable generator. In particular, a single generator is implemented using adaptive instance normalization (AdaIN) layers so that the baseline generator converting a low-dose CT image to a routine-dose CT image can be switched to a generator converting high-dose to low-dose by simply changing the AdaIN code. Thanks to the shared baseline network, the additional memory requirement and weight increases are minimized, and the training can be done more stably even with small training data. Experimental results show that the proposed method outperforms the previous cycleGAN approaches while using only about half the parameters.

[1]  Timo Aila,et al.  A Style-Based Generator Architecture for Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Jong Chul Ye,et al.  Optimal Transport, CycleGAN, and Penalized LS for Unsupervised Learning in Inverse Problems , 2019, ArXiv.

[4]  Max A. Viergever,et al.  Generative Adversarial Networks for Noise Reduction in Low-Dose CT , 2017, IEEE Transactions on Medical Imaging.

[5]  Feng Lin,et al.  Low-Dose CT With a Residual Encoder-Decoder Convolutional Neural Network , 2017, IEEE Transactions on Medical Imaging.

[6]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[7]  Dong Hyun Yang,et al.  Demonstration of mitral valve prolapse with CT for planning of mitral valve repair. , 2014, Radiographics : a review publication of the Radiological Society of North America, Inc.

[8]  Jong Chul Ye,et al.  A deep convolutional neural network using directional wavelets for low‐dose X‐ray CT reconstruction , 2016, Medical physics.

[9]  J. Leipsic,et al.  Adaptive statistical iterative reconstruction: assessment of image noise and image quality in coronary CT angiography. , 2010, AJR. American journal of roentgenology.

[10]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[11]  Stephen M. Moore,et al.  The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository , 2013, Journal of Digital Imaging.

[12]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[14]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[15]  Minh N. Do,et al.  Nonsubsampled contourlet transform: construction and application in enhancement , 2005, IEEE International Conference on Image Processing 2005.

[16]  C. Villani Optimal Transport: Old and New , 2008 .

[17]  Dongjin Han,et al.  Stress Myocardial Perfusion CT in Patients Suspected of Having Coronary Artery Disease: Visual and Quantitative Analysis-Validation by Using Fractional Flow Reserve. , 2015, Radiology.

[18]  Serge J. Belongie,et al.  Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[19]  Daniel Kolditz,et al.  Iterative reconstruction methods in X-ray CT. , 2012, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[20]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[21]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Youssef Mroueh,et al.  Wasserstein Style Transfer , 2019, AISTATS.

[23]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[24]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[25]  Xinhui Duan,et al.  Technical Note: Development and validation of an open data format for CT projection data. , 2015, Medical physics.

[26]  C. Fink,et al.  Evaluation of heavily calcified vessels with coronary CT angiography: comparison of iterative and filtered back projection image reconstruction. , 2011, Radiology.

[27]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[28]  Jong Chul Ye,et al.  Cycle‐consistent adversarial denoising network for multiphase coronary CT angiography , 2018, Medical physics.

[29]  Xuanqin Mou,et al.  Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss , 2017, IEEE Transactions on Medical Imaging.

[30]  Jaejun Yoo,et al.  Deep Convolutional Framelet Denosing for Low-Dose CT via Wavelet Residual Network , 2017, IEEE Transactions on Medical Imaging.

[31]  Jong Chul Ye,et al.  Unsupervised Denoising for Satellite Imagery using Wavelet Subband CycleGAN , 2020, ArXiv.

[32]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[33]  Gabriel Peyré,et al.  Computational Optimal Transport , 2018, Found. Trends Mach. Learn..

[34]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.