Auto-contouring FDG-PET/MR images for cervical cancer radiation therapy: An intelligent sequential approach using focally trained, shallow U-Nets

Abstract Background Manual contouring for radiation therapy planning remains the most laborious and time consuming part in the radiation therapy workflow. Particularly for cervical cancer, this task is complicated by the complex female pelvic anatomy and the concomitant dependence on 18F-labeled Fluorodeoxyglucose (FDG) positron emission tomography (PET) and magnetic resonance (MR) images. Using deep learning, we propose a new auto-contouring method for FDG-PET/MR based cervical cancer radiation therapy by combining the high level anatomical topography and radiological properties, to the low-level pixel wise deep-learning based semantic segmentation. Materials/methods The proposed method: 1) takes advantage of PET data and left/right anatomical symmetry, creating sub-volumes that are centered on the structures to be contoured. 2) Uses a 3D shallow U-Net (sU-Net) model with an encoder depth of 2.3) Applies the successive training of 3 consecutive sU-Nets in a feed forward strategy. 4) Employs, instead of the usual generalized dice loss function (GDL), a patch dice loss function (PDL) that takes into account the Dice similarity index (DSI) at the level of each training patch. Experimental analysis was conducted on a set of 13 PET/MR images was using a leave-one-out strategy. Results Despite the limited data availability, 5 anatomical structures - the gross tumor volume, bladder, anorectum, and bilateral femurs - were accurately (DSI ​= ​0.78), rapidly (1.9 ​s/structure), and automatically delineated by our algorithm. Overall, PDL achieved a better performance than GDL and DSI was higher for organs at risk (OARs) with solid tissue (e.g. femurs) than for OARs with air-filled soft tissues (e.g. anorectum). Conclusion The presented workflow successfully addresses the challenge of auto-contouring in FDG-PET/MR based cervical cancer. It is expected to expedite the cervical cancer radiation therapy workflow in both, conventional and adaptive radiation therapy settings.

[1]  Loïc Le Folgoc,et al.  Attention U-Net: Learning Where to Look for the Pancreas , 2018, ArXiv.

[2]  B. Erickson,et al.  Appropriate magnetic resonance imaging techniques for gross tumor volume delineation in external beam radiation therapy of locally advanced cervical cancer , 2015, Oncotarget.

[3]  Paul F. Whelan,et al.  Convolutional neural network on three orthogonal planes for dynamic texture classification , 2017, Pattern Recognit..

[4]  Susanne Heinzer,et al.  Sequential whole-body PET/MR scanner: concept, clinical use, and optimisation after two years in the clinic. The manufacturer’s perspective , 2013, Magnetic Resonance Materials in Physics, Biology and Medicine.

[5]  Atallah Baydoun,et al.  Abdominal, multi-organ, auto-contouring method for online adaptive magnetic resonance guided radiotherapy: An intelligent, multi-level fusion approach , 2018, Artif. Intell. Medicine.

[6]  G. Rauch,et al.  Optimization of MR imaging for pretreatment evaluation of patients with endometrial and cervical cancer. , 2014, Radiographics : a review publication of the Radiological Society of North America, Inc.

[7]  Appropriate Methodology for EBRT and HDR Intracavitary/Interstitial Brachytherapy Dose Composite and Clinical Plan Evaluation for Patients with Cervical Cancer. , 2019, Practical radiation oncology.

[8]  Tian Liu,et al.  Automatic multiorgan segmentation in thorax CT images using U-net-GAN. , 2019, Medical physics.

[9]  Bradley J. Erickson,et al.  RIL-Contour: a Medical Imaging Dataset Annotation Tool for and with Deep Learning , 2019, Journal of Digital Imaging.

[10]  D. Heron,et al.  What the diagnostic radiologist needs to know about radiation oncology. , 2011, Radiology.

[11]  Michael G Jameson,et al.  A review of methods of analysis in contouring studies for radiation oncology , 2010, Journal of medical imaging and radiation oncology.

[12]  Sébastien Ourselin,et al.  Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations , 2017, DLMIA/ML-CDS@MICCAI.

[13]  Jose Dolz,et al.  Multi-Scale Self-Guided Attention for Medical Image Segmentation , 2021, IEEE Journal of Biomedical and Health Informatics.

[14]  J Hennig,et al.  RARE imaging: A fast imaging method for clinical MR , 1986, Magnetic resonance in medicine.

[15]  R. Knobler,et al.  Fast tissue segmentation based on a 4D feature map in characterization of intracranial lesions , 1999, Journal of magnetic resonance imaging : JMRI.

[16]  Yu Song,et al.  The influence of automation on tumor contouring , 2017, Cognition, Technology & Work.

[17]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[18]  A. Grosu,et al.  Target Volume Definition in Radiation Oncology , 2015 .

[19]  Clifton D Fuller,et al.  Deep Learning Algorithm for Auto-Delineation of High-Risk Oropharyngeal Clinical Target Volumes With Built-In Dice Similarity Coefficient Parameter Optimization Function. , 2018, International journal of radiation oncology, biology, physics.

[20]  Lotfi A. Zadeh,et al.  Toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logic , 1997, Fuzzy Sets Syst..

[21]  A. Jemal,et al.  Cancer statistics, 2020 , 2020, CA: a cancer journal for clinicians.

[22]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[23]  Guang Yang,et al.  Automatic Brain Tumor Detection and Segmentation Using U-Net Based Fully Convolutional Networks , 2017, MIUA.

[24]  Gabriela Csurka,et al.  What is a good evaluation measure for semantic segmentation? , 2013, BMVC.

[25]  V. Budach,et al.  Protection of quality and innovation in radiation oncology: The prospective multicenter trial the German Society of Radiation Oncology (DEGRO-QUIRO study) , 2014, Strahlentherapie und Onkologie.

[26]  Brent van der Heyden,et al.  Evaluation of measures for assessing time-saving of automatic organ-at-risk segmentation in radiotherapy , 2019, Physics and imaging in radiation oncology.

[27]  Martin Tomko,et al.  Infrastructure-Independent Indoor Localization and Navigation , 2019, ACM Comput. Surv..

[28]  Tao Zhang,et al.  Fully automatic and robust segmentation of the clinical target volume for radiotherapy of breast cancer using big data and deep learning. , 2018, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[29]  Z. Zeng,et al.  A knowledge‐based approach to automated planning for hepatocellular carcinoma , 2017, Journal of applied clinical medical physics.

[30]  Zhikai Liu,et al.  Segmentation of organs-at-risk in cervical cancer CT images with a convolutional neural network. , 2020, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[31]  Steve B. Jiang,et al.  Fully automated organ segmentation in male pelvic CT images , 2018, Physics in medicine and biology.

[32]  R F Muzic,et al.  COMKAT: compartment model kinetic analysis tool. , 2001, Journal of nuclear medicine : official publication, Society of Nuclear Medicine.

[33]  Jacques Ferlay,et al.  Estimates of incidence and mortality of cervical cancer in 2018: a worldwide analysis , 2019, The Lancet. Global health.

[34]  Pavel Král,et al.  Deep Generalized Max Pooling , 2019, 2019 International Conference on Document Analysis and Recognition (ICDAR).

[35]  S. Zincirkeser,et al.  Standardized Uptake Values of Normal Organs on 18F-Fluorodeoxyglucose Positron Emission Tomography and Computed Tomography Imaging , 2007, The Journal of international medical research.

[36]  Camille Couprie,et al.  Semantic Segmentation using Adversarial Networks , 2016, NIPS 2016.

[37]  B. Erickson,et al.  Radiation Therapy for Cervical Cancer: Executive Summary of an ASTRO Clinical Practice Guideline. , 2020, Practical radiation oncology.

[38]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[39]  Hsuan-Ming Huang,et al.  Integrated Software Environment Based on COMKAT for Analyzing Tracer Pharmacokinetics with Molecular Imaging , 2010, Journal of Nuclear Medicine.

[40]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[41]  Hongzan Sun,et al.  Defining PET tumor volume in cervical cancer with hybrid PET/MRI: a comparative study , 2014, Nuclear medicine communications.

[42]  Issam El Naqa,et al.  Pelvic normal tissue contouring guidelines for radiation therapy: a Radiation Therapy Oncology Group consensus panel atlas. , 2012, International journal of radiation oncology, biology, physics.

[43]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[44]  Ujjwal Maulik,et al.  Understanding Deep Learning Techniques for Image Segmentation , 2019, ACM Comput. Surv..

[45]  Atallah Baydoun,et al.  mDixon-Based Synthetic CT Generation for PET Attenuation Correction on Abdomen and Pelvis Jointly Using Transfer Fuzzy Clustering and Active Learning-Based Classification , 2020, IEEE Transactions on Medical Imaging.

[46]  Liyuan Chen,et al.  Automatic PET cervical tumor segmentation by combining deep learning and anatomic prior , 2019, Physics in medicine and biology.

[47]  Atallah Baydoun,et al.  Estimating CT from MR Abdominal Images Using Novel Generative Adversarial Networks , 2020, Journal of Grid Computing.

[48]  Keiron O'Shea,et al.  An Introduction to Convolutional Neural Networks , 2015, ArXiv.

[49]  Olivier Alata,et al.  Unsupervised segmentation for automatic detection of brain tumors in MRI , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[50]  Gernot A. Fink,et al.  PHOCNet: A Deep Convolutional Neural Network for Word Spotting in Handwritten Documents , 2016, 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR).

[51]  H. Zaidi,et al.  Design and performance evaluation of a whole-body Ingenuity TF PET–MRI system , 2011, Physics in medicine and biology.

[52]  Tareq Abed Mohammed,et al.  Understanding of a convolutional neural network , 2017, 2017 International Conference on Engineering and Technology (ICET).

[53]  T. Yen,et al.  Deep learning for fully automated tumor segmentation and extraction of magnetic resonance radiomics features in cervical cancer , 2019, European Radiology.

[54]  Lu Wang,et al.  An Improved Dice Loss for Pneumothorax Segmentation by Mining the Information of Negative Areas , 2020, IEEE Access.

[55]  Michael Lock,et al.  Technology assessment of automated atlas based segmentation in prostate bed contouring , 2011, Radiation oncology.

[56]  E. Sala,et al.  Staging of uterine cervical cancer with MRI: guidelines of the European Society of Urogenital Radiology , 2011, European Radiology.

[57]  Carbon ion reirradiation compared to intensity-modulated re-radiotherapy for recurrent head and neck cancer (CARE): a randomized controlled trial , 2020, Radiation oncology.

[58]  P. Jaccard,et al.  Etude comparative de la distribution florale dans une portion des Alpes et des Jura , 1901 .

[59]  Christoph Meinel,et al.  Conditional Generative Refinement Adversarial Networks for Unbalanced Medical Image Semantic Segmentation , 2018, ArXiv.

[60]  Kathleen R. Cho,et al.  Cervical Cancer, Version 3.2019, NCCN Clinical Practice Guidelines in Oncology. , 2019, Journal of the National Comprehensive Cancer Network : JNCCN.

[61]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[62]  Y. Kim,et al.  Atlas-based auto-segmentation for postoperative radiotherapy planning in endometrial and cervical cancers , 2020, Radiation Oncology.

[63]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[64]  Luca Boldrini,et al.  Deep Learning: A Review for the Radiation Oncologist , 2019, Front. Oncol..

[65]  C. Pal,et al.  Deep Learning: A Primer for Radiologists. , 2017, Radiographics : a review publication of the Radiological Society of North America, Inc.

[66]  Luca Maria Gambardella,et al.  Max-pooling convolutional neural networks for vision-based hand gesture recognition , 2011, 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[67]  Hongzan Sun,et al.  Comparison of Tumor Volume Between PET and MRI in Cervical Cancer With Hybrid PET/MR , 2014, International Journal of Gynecologic Cancer.

[68]  Yongsook C. Lee,et al.  Clinical response of pelvic and para-aortic lymphadenopathy to a radiation boost in the definitive management of locally advanced cervical cancer. , 2013, International journal of radiation oncology, biology, physics.

[69]  Antonio J. Plaza,et al.  Image Segmentation Using Deep Learning: A Survey , 2021, IEEE transactions on pattern analysis and machine intelligence.

[70]  Ron Kikinis,et al.  Statistical validation of image segmentation quality based on a spatial overlap index. , 2004, Academic radiology.

[71]  Paul Aljabar,et al.  Clinical evaluation of atlas and deep learning based automatic contouring for lung cancer. , 2017, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[72]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[73]  H. Sebastian Seung,et al.  Permitted and Forbidden Sets in Symmetric Threshold-Linear Networks , 2003, Neural Computation.

[74]  Yujian Li,et al.  A concatenating framework of shortcut convolutional neural networks , 2017, ArXiv.

[75]  G. Angelelli,et al.  Standard high-resolution pelvic MRI vs. low-resolution pelvic MRI in the evaluation of deep infiltrating endometriosis , 2014, European Radiology.

[76]  R R Edelman,et al.  Half-fourier acquisition single-shot turbo spin-echo (HASTE) MR: comparison with fast spin-echo MR in diseases of the brain. , 1997, AJNR. American journal of neuroradiology.