Stepwise Local Synthetic Pseudo-CT Imaging Based on Anatomical Semantic Guidance

In this study, an anatomic semantic guided neural style transfer (ASGNST) algorithm was developed and pseudo-computed tomography (CT) images synthesized in steps. CT images and ultrasound (US) images of 20 cervical cancer patients to be treated were selected. The foreground (FG) and background (BG) regions of the US images were segmented by the region growth method, and three objective functions for content, style, and contour loss were defined. Based on the two types of regions, a local pseudo-CT image synthesis model based on a convolution neural network was established. Then, global 2D pseudo-CT images were obtained using the weighted average fusing algorithm, and the final pseudo-CT images were obtained through 3D reconstruction. US phantom and data of five additional cervical cancer patients were used for prediction. Furthermore, three image synthesis algorithms—global deformation field (GDF), stepwise local deformation field (SLDF), and neural style transfer (NST)—were selected for comparative verification. The pseudo-CT images synthesized by the four algorithms were compared with the ground-truth CT images obtained during treatment. The structural similarity index between the ground-truth CT and pseudo-CT synthesized by the improved algorithm significantly differed from those synthesized by the other three algorithms (<inline-formula> <tex-math notation="LaTeX">$\text{t}_{\mathrm {GDF\_{}bg}}=7.175$ </tex-math></inline-formula>, <inline-formula> <tex-math notation="LaTeX">$\text{t}_{\mathrm {SLDF\_{}bg}}=4.513$ </tex-math></inline-formula>, <inline-formula> <tex-math notation="LaTeX">$\text{t}_{\mathrm {NST\_{}bg}}=3.228$ </tex-math></inline-formula>, <inline-formula> <tex-math notation="LaTeX">$\text{t}_{\mathrm {GDF\_{}fg}}=10.518$ </tex-math></inline-formula>, <inline-formula> <tex-math notation="LaTeX">$\text{t}_{\mathrm {SLDF\_{}fg}}=5.522$ </tex-math></inline-formula>, <inline-formula> <tex-math notation="LaTeX">$\text{t}_{\mathrm {NST\_{}fg}}=2.869$ </tex-math></inline-formula>, p < 0.05). Further, the mean absolute error and peak signal-to-noise ratio values prove that the pseudo-CT synthesized by the ASGNST algorithm is similar to the ground-truth CT. The improved algorithm can obtain pseudo-CT images with high precision and provides a novel direction for image guidance in cervical cancer brachytherapy.

[1]  C. Kirisits,et al.  Adaptive image guided brachytherapy for cervical cancer: A combined MRI-/CT-planning technique with MRI only at first fraction , 2013, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[2]  Yu-Kun Lai,et al.  Automatic semantic style transfer using deep convolutional neural networks and soft masks , 2017, The Visual Computer.

[3]  Chuan Li,et al.  Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Youbao Tang,et al.  Scene Text Detection and Segmentation Based on Cascaded Convolution Neural Networks , 2017, IEEE Transactions on Image Processing.

[5]  Yaozong Gao,et al.  Region-Adaptive Deformable Registration of CT/MRI Pelvic Images via Learning-Based Image Synthesis , 2018, IEEE Transactions on Image Processing.

[6]  Linda Doyle,et al.  Painting style transfer for head portraits using convolutional neural networks , 2016, ACM Trans. Graph..

[7]  W. Majewski,et al.  Dose distribution in bladder and surrounding normal tissues in relation to bladder volume in conformal radiotherapy for bladder cancer. , 2009, International journal of radiation oncology, biology, physics.

[8]  Dinggang Shen,et al.  Medical Image Synthesis with Deep Convolutional Adversarial Networks , 2018, IEEE Transactions on Biomedical Engineering.

[9]  Yi Wang,et al.  Dose-volume parameters and clinical outcome of CT-guided free-hand high-dose-rate interstitial brachytherapy for cervical cancer , 2012, Chinese journal of cancer.

[10]  Koen Van Leemput,et al.  A patch-based pseudo-CT approach for MRI-only radiotherapy in the pelvis. , 2016, Medical physics.

[11]  Alex J. Champandard,et al.  Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artworks , 2016, ArXiv.

[12]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[13]  Jean Ponce,et al.  Learning mid-level features for recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Gang Hua,et al.  Visual attribute transfer through deep image analogy , 2017, ACM Trans. Graph..

[15]  Fang Wang,et al.  Comparison of computed tomography and magnetic resonance imaging in cervical cancer brachytherapy: A systematic review. , 2017, Brachytherapy.

[16]  Yaozong Gao,et al.  Dual‐core steered non‐rigid registration for multi‐modal images via bi‐directional image synthesis , 2017, Medical Image Anal..

[17]  Andrea Vedaldi,et al.  Texture Networks: Feed-forward Synthesis of Textures and Stylized Images , 2016, ICML.

[18]  Hao Wang,et al.  Real-Time Neural Style Transfer for Videos , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Davide Fontanarosa,et al.  Critical assessment of intramodality 3D ultrasound imaging for prostate IGRT compared to fiducial markers. , 2013, Medical physics.

[20]  W P Dillon,et al.  Reducing Patient Radiation Dose during CT-Guided Procedures: Demonstration in Spinal Injections for Pain , 2011, American Journal of Neuroradiology.

[21]  F. Verhaegen,et al.  Various approaches for pseudo-CT scan creation based on ultrasound to ultrasound deformable image registration between different treatment time points for radiotherapy treatment plan adaptation in prostate cancer patients , 2016 .

[22]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[23]  Hao Li,et al.  Anime Style Space Exploration Using Metric Learning and Generative Adversarial Networks , 2018, ArXiv.

[24]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[25]  Cervical cancer brachytherapy in Canada: A focus on interstitial brachytherapy utilization. , 2016, Brachytherapy.

[26]  Leon A. Gatys,et al.  A Neural Algorithm of Artistic Style , 2015, ArXiv.

[27]  Tao Lin,et al.  Imaging study of pseudo-CT images of superposed ultrasound deformation fields acquired in radiotherapy based on step-by-step local registration , 2018, Medical & Biological Engineering & Computing.

[28]  Alireza Bab-Hadiashar,et al.  Convolutional neural networks for texture recognition using transfer learning , 2017, 2017 International Conference on Control, Automation and Information Sciences (ICCAIS).

[29]  Davide Fontanarosa,et al.  Simulation of pseudo-CT images based on deformable image registration of ultrasound images: A proof of concept for transabdominal ultrasound imaging of the prostate during radiotherapy. , 2016, Medical physics.

[30]  Phillip M. Cheng,et al.  Transfer Learning with Convolutional Neural Networks for Classification of Abdominal Ultrasound Images , 2017, Journal of Digital Imaging.