Training deep cross-modality conversion models with a small amount of data and its application to MVCT to kVCT conversion

Purpose: Deep-learning-based image processing has emerged as a valuable tool in recent years owing to its high performance. However, the quality of deep-learning-based methods relies heavily on the amount of training data, and the cost of acquiring a large amount of data is often prohibitive in medical fields. Therefore, we performed CT modality conversion based on deep learning requiring only a small number of unsupervised images. Methods: The proposed method is based on generative adversarial networks (GANs) with several extensions tailored for CT images. This method emphasizes the preservation of the structure in the processed images and reduction in the amount of training data. This method was applied to realize the conversion of mega-voltage computed tomography (MVCT) to kilo-voltage computed tomography (kVCT) images. Training was performed using several datasets acquired from patients with head and neck cancer. The size of the datasets ranged from 16 slices (for two patients) to 2745 slices (for 137 patients) of MVCT and 2824 slices of kVCT for 98 patients. Results: The quality of the processed MVCT images was considerably enhanced, and the structural changes in the images were minimized. With an increase in the size of training data, the image quality exhibited a satisfactory convergence from a few hundred slices. In addition to statistical and visual evaluations, these results were clinically evaluated by medical doctors in terms of the accuracy of contouring. Conclusion: We developed an MVCT to kVCT conversion model based on deep learning, which can be trained using a few hundred unpaired images. The stability of the model against the change in the data size was demonstrated. This research promotes the reliable use of deep learning in clinical medicine by partially answering the commonly asked questions: “Is our data enough? How much data must we prepare?” ∗ E-mail: shoozaki0117@gmail.com

[1]  Tadaaki Kirita,et al.  Regularized Three-Dimensional Generative Adversarial Nets for Unsupervised Metal Artifact Reduction in Head and Neck CT Images , 2020, IEEE Access.

[2]  Peter R Seevinck,et al.  Dose evaluation of fast synthetic-CT generation using a generative adversarial network for general pelvis MR-only radiotherapy , 2018, Physics in medicine and biology.

[3]  Takahiro Nakamoto,et al.  Fast Statistical Iterative Reconstruction for Mega-voltage Computed Tomography. , 2020, The journal of medical investigation : JMI.

[4]  T. Mackie,et al.  Megavoltage CT on a tomotherapy system. , 1999, Physics in medicine and biology.

[5]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[6]  Patrick A Kupelian,et al.  Performance characterization of megavoltage computed tomography imaging on a helical tomotherapy unit. , 2005, Medical physics.

[7]  J O Deasy,et al.  Tomotherapy: a new concept for the delivery of dynamic conformal radiotherapy. , 1993, Medical physics.

[8]  Kirby G. Vosburgh,et al.  3D Slicer: A Platform for Subject-Specific Image Analysis, Visualization, and Clinical Support , 2014 .

[9]  Yang Lei,et al.  A review on medical imaging synthesis using deep learning and its clinical applications , 2020, Journal of applied clinical medical physics.

[10]  Jelmer M. Wolterink,et al.  Deep MR to CT Synthesis Using Unpaired Data , 2017, SASHIMI@MICCAI.

[11]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[12]  Alexander A. Zamyatin,et al.  Metal artifact reduction algorithm for single energy and dual energy CT scans , 2012, 2012 IEEE Nuclear Science Symposium and Medical Imaging Conference Record (NSS/MIC).

[13]  A. Blum,et al.  Total hip prosthesis CT with single-energy projection-based metallic artifact reduction: impact on the visualization of specific periprosthetic soft tissue structures , 2014, Skeletal Radiology.

[14]  S. Kida,et al.  Visual enhancement of Cone-beam CT by use of CycleGAN. , 2019, Medical physics.

[15]  T R Mackie,et al.  History of tomotherapy , 2006, Physics in medicine and biology.

[16]  Shizuo Kaji,et al.  Overview of image-to-image translation by use of deep neural networks: denoising, super-resolution, modality conversion, and reconstruction in medical imaging , 2019, Radiological Physics and Technology.

[17]  A. Sudhyadhom,et al.  Improved contrast and noise of megavoltage computed tomography (MVCT) through cycle-consistent generative machine learning. , 2020, Medical physics.

[18]  K. Parodi,et al.  Developments in deep learning based corrections of cone beam computed tomography to enable dose calculations for adaptive radiotherapy , 2020, Physics and imaging in radiation oncology.

[19]  Jing Wang,et al.  Generating synthesized computed tomography (CT) from cone-beam computed tomography (CBCT) using CycleGAN for adaptive radiation therapy , 2018, Physics in medicine and biology.

[20]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[21]  B. Kavanagh,et al.  High-dose MVCT image guidance for stereotactic body radiation therapy. , 2012, Medical physics.

[22]  Paolo Zaffino,et al.  Deep learning-based synthetic-CT generation in radiotherapy and PET: a review , 2021, Medical physics.