GANai: Standardizing CT Images using Generative Adversarial Network with Alternative Improvement

Computed tomography (CT) is a widely-used diag-reproducibility regarding radiomic features, such as intensity, nostic image modality routinely used for assessing anatomical tissue characteristics. However, non-standardized imaging pro-tocols are commonplace, which poses a fundamental challenge in large-scale cross-center CT image analysis. One approach to address the problem is to standardize CT images using generative adversarial network models (GAN). GAN learns the data distribution of training images and generate synthesized images under the same distribution. However, existing GAN models are not directly applicable to this task mainly due to the lack of constraints on the mode of data to generate. Furthermore, they treat every image equally, but in real applications, some images are more difficult to standardize than the others. All these may lead to the lack-of-detail problem in CT image synthesis. We present a new GAN model called GANai to mitigate the differences in radiomic features across CT images captured using non-standard imaging protocols. Given source images, GANai composes new images by specifying a high-level goal that the image features of the synthesized images should be similar to those of the standard images. GANai introduces an alternative improvement training strategy to alternatively and steadily improve model performance. The new training strategy enables a series of technical improvements, including phase-specific loss functions, phase-specific training data, and the adoption of ensemble learning, leading to better model performance. The experimental results show that GANai is significantly better than the existing state-of-the-art image synthesis algorithms on CT image standardization. Also, it significantly improves the efficiency and stability of GAN model training.

[1]  V T Farewell,et al.  Use of risk-adjusted CUSUM and RSPRTcharts for monitoring in medical contexts , 2003, Statistical methods in medical research.

[2]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.

[3]  K. Thayalan The Physics of Radiology and Imaging , 2014 .

[4]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[5]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[6]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[7]  C. McCollough,et al.  Relationship between noise, dose, and pitch in cardiac multi-detector row CT. , 2006, Radiographics : a review publication of the Radiological Society of North America, Inc.

[8]  H. Aerts Semantics Features : Phenotype Quantification by a Radiologist ’ s Expert Eye , 2016 .

[9]  Anil K. Jain Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[10]  Sridhar Mahadevan,et al.  Generative Multi-Adversarial Networks , 2016, ICLR.

[11]  David Pfau,et al.  Connecting Generative Adversarial Networks and Actor-Critic Methods , 2016, ArXiv.

[12]  E. Arredondo,et al.  Council of Europe Black Sea Area Project: International Cooperation for the Development of Activities Related to Donation and Transplantation of Organs in the Region. , 2018, Transplantation proceedings.

[13]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[14]  Kujtim Latifi,et al.  Imaging features from pretreatment CT scans are associated with clinical outcomes in nonsmall‐cell lung cancer patients treated with stereotactic body radiotherapy , 2017, Medical physics.

[15]  Paul Suetens Fundamentals of Medical Imaging , 2002 .

[16]  E. S. Page CONTINUOUS INSPECTION SCHEMES , 1954 .

[17]  O. Mawlawi,et al.  Prognostic value and reproducibility of pretreatment CT texture features in stage III non-small cell lung cancer. , 2014, International journal of radiation oncology, biology, physics.

[18]  Bernt Schiele,et al.  Generative Adversarial Text to Image Synthesis , 2016, ICML.

[19]  Andre Dekker,et al.  Radiogenomics: radiobiology enters the era of big data and team science. , 2014, International journal of radiation oncology, biology, physics.

[20]  Gözde B. Ünal,et al.  Patch-Based Image Inpainting with Generative Adversarial Networks , 2018, ArXiv.

[21]  Mahadevappa Mahesh Fundamentals of Medical Imaging, 2nd Edition , 2011 .

[22]  J. Canales‐Vázquez,et al.  Radiomics of CT Features May Be Nonreproducible and Redundant: Influence of CT Acquisition Parameters. , 2018, Radiology.

[23]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[24]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[25]  E. Fishman,et al.  CT scan parameters and radiation dose: practical advice for radiologists. , 2013, Journal of the American College of Radiology : JACR.

[26]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[27]  Mithat Gönen,et al.  Influence of CT acquisition and reconstruction parameters on radiomic feature reproducibility , 2018, Journal of medical imaging.

[28]  Azriel Rosenfeld,et al.  Digital picture analysis , 1976 .

[29]  Joost van de Weijer,et al.  Ensembles of Generative Adversarial Networks , 2016, ArXiv.

[30]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[31]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[32]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[33]  Jie Zhang,et al.  The ABCs of Accurate Volumetric Measurement of Cerebral Hematoma , 2011, Stroke.

[34]  Michael F. Cohen,et al.  Radiosity and realistic image synthesis , 1993 .

[35]  Karin Schwab,et al.  Medical Imaging Signals And Systems , 2016 .

[36]  Zhi-pei Liang,et al.  Introduction to biomedical imaging , 2008 .

[37]  Camille Couprie,et al.  Semantic Segmentation using Adversarial Networks , 2016, NIPS 2016.

[38]  Tinsu Pan,et al.  Fundamentals of Medical Imaging , 2010, The Journal of Nuclear Medicine.

[39]  M. Martel,et al.  High quality machine-robust image features: identification in nonsmall cell lung cancer computed tomography images. , 2013, Medical physics.

[40]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[41]  Hayit Greenspan,et al.  Cross-Modality Synthesis from CT to PET using FCN and GAN Networks for Improved Automated Lesion Detection , 2018, Eng. Appl. Artif. Intell..

[42]  E. Dubois,et al.  Digital picture processing , 1985, Proceedings of the IEEE.

[43]  H. Aerts,et al.  Applications and limitations of radiomics , 2016, Physics in medicine and biology.

[44]  Hyunsoo Kim,et al.  Learning to Discover Cross-Domain Relations with Generative Adversarial Networks , 2017, ICML.

[45]  Christin Wirth The Essential Physics of Medical Imaging , 2003, European Journal of Nuclear Medicine and Molecular Imaging.

[46]  Huaiyu Zhu On Information and Sufficiency , 1997 .

[47]  Arthur R. Weeks,et al.  Histogram specification of 24-bit color images in the color difference (C-Y) color space , 1999, J. Electronic Imaging.

[48]  Yann LeCun,et al.  Energy-based Generative Adversarial Network , 2016, ICLR.

[49]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  D. Sullivan,et al.  A collaborative enterprise for multi-stakeholder participation in the advancement of quantitative imaging. , 2011, Radiology.