Continuous Conversion of CT Kernel Using Switchable CycleGAN With AdaIN

X-ray computed tomography (CT) uses different filter kernels to highlight different structures. Since the raw sinogram data is usually removed after the reconstruction, in case there is additional need for other types of kernel images that were not previously generated, the patient may need to be scanned again. Accordingly, there exists increasing demand for post-hoc image domain conversion from one kernel to another without sacrificing the image quality. In this paper, we propose a novel unsupervised continuous kernel conversion method using cycle-consistent generative adversarial network (cycleGAN) with adaptive instance normalization (AdaIN). Even without paired training data, not only can our network translate the images between two different kernels, but it can also convert images along the interpolation path between the two kernel domains. We also show that the quality of generated images can be further improved if intermediate kernel domain images are available. Experimental results confirm that our method not only enables accurate kernel conversion that is comparable to supervised learning methods, but also generates intermediate kernel images in the unseen domain that are useful for hypopharyngeal cancer diagnosis.

[1]  Zohair Al-Ameen,et al.  Reducing the gaussian blur artifact from ct medical images by employing a combination of sharpening filters and iterative deblurring algorithms , 2012 .

[2]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Andrey Nino Kurniawan,et al.  The Optimization of Mastoid CT Image Using Windows and Kernel Reconstructions , 2020 .

[4]  Youssef Mroueh,et al.  Wasserstein Style Transfer , 2019, AISTATS.

[5]  C. Villani Optimal Transport: Old and New , 2008 .

[6]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[7]  Philip F. Judy,et al.  Normalizing computed tomography data reconstructed with different filter kernels: effect on emphysema quantification , 2015, European Radiology.

[8]  Dongmei Sun,et al.  Hybrid convolution kernel: optimized CT of the head, neck, and spine. , 2011, AJR. American journal of roentgenology.

[9]  Gabriel Peyré,et al.  Computational Optimal Transport , 2018, Found. Trends Mach. Learn..

[10]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[11]  Sang Min Lee,et al.  CT Image Conversion among Different Reconstruction Kernels without a Sinogram by Using a Convolutional Neural Network , 2018, Korean journal of radiology.

[12]  Shuai Leng,et al.  Synthesizing images from multiple kernels using a deep convolutional neural network. , 2019, Medical physics.

[13]  Jonathan G Goldin,et al.  Emphysema: effect of reconstruction algorithm on CT imaging measures. , 2004, Radiology.

[14]  Serge J. Belongie,et al.  Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[15]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[16]  S. Sanada,et al.  Combined multi-kernel head computed tomography images optimized for depicting both brain parenchyma and bone. , 2014, Journal of X-ray science and technology.

[17]  Sang Min Lee,et al.  Deep Learning-based Image Conversion of CT Reconstruction Kernels Improves Radiomics Reproducibility for Pulmonary Nodules or Masses. , 2019, Radiology.

[18]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[19]  Jong Chul Ye,et al.  Deep Learning COVID-19 Features on CXR Using Limited Training Data Sets , 2020, IEEE Transactions on Medical Imaging.

[20]  Jong Chul Ye,et al.  AdaIN-Switchable CycleGAN for Efficient Unsupervised Low-Dose CT Denoising , 2020, ArXiv.

[21]  Jong Chul Ye,et al.  Mumford–Shah Loss Functional for Image Segmentation With Deep Learning , 2019, IEEE Transactions on Image Processing.

[22]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[23]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.