Fully automatic catheter segmentation in MRI with 3D convolutional neural networks: application to MRI-guided gynecologic brachytherapy

External-beam radiotherapy followed by High Dose Rate (HDR) brachytherapy is the standard-of-care for treating gynecologic cancers. The enhanced soft-tissue contrast provided by Magnetic Resonance Imaging (MRI) makes it a valuable imaging modality for diagnosing and treating these cancers. However, in contrast to Computed Tomography (CT) imaging, the appearance of the brachytherapy catheters, through which radiation sources are inserted to reach the cancerous tissue later on, is often variable across images. This paper reports, for the first time, a new deep-learning-based method for fully automatic segmentation of multiple closely spaced brachytherapy catheters in intraoperative MRI. Represented in the data are 50 gynecologic cancer patients treated by MRI-guided HDR brachytherapy. For each patient, a single intraoperative MRI was used. 826 catheters in the images were manually segmented by an expert radiation physicist who is also a trained radiation oncologist. The number of catheters in a patient ranged between 10 and 35. A deep 3-dimensional Convolutional Neural Network (CNN) model was developed and trained. In order to make the learning process more robust, the network was trained 5 times, each time using a different combination of shown patients. Finally, each test case was processed by the 5 networks and the final segmentation was generated by voting on the obtained 5 candidate segmentations. 4-fold validation was executed and all the patients were segmented. An average distance error of 2.0±3.4 mm was achieved. False positive and false negative catheters were 6.7% and 1.5% respectively. Average Dice score was equal to 0.60±0.17. The algorithm is available for use in the open source software platform 3D Slicer allowing for wide scale testing and research discussion. In conclusion, to the best of our knowledge, fully automatic segmentation of multiple closely spaced catheters from intraoperative MR images was achieved for the first time in gynecological brachytherapy.

[1]  A Wambersie,et al.  Comparison of radiography- and computed tomography-based treatment planning in cervix cancer in brachytherapy with specific attention to some quality assurance aspects. , 2001, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[2]  Christian Kirisits,et al.  Recommendations from Gynaecological (GYN) GEC-ESTRO Working Group (I): concepts and terms in 3D image based 3D treatment planning in cervix cancer brachytherapy with emphasis on MRI assessment of GTV and CTV. , 2005, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[3]  Ross T. Whitaker,et al.  Improved segmentation of white matter tracts with adaptive Riemannian metrics , 2014, Medical Image Anal..

[4]  Bruce Thomadsen,et al.  American Brachytherapy Society consensus guidelines for locally advanced carcinoma of the cervix. Part I: general principles. , 2012, Brachytherapy.

[5]  Purang Abolmaesumi,et al.  DeepInfer: open-source deep learning deployment toolkit for image-guided therapy , 2017, Medical Imaging.

[6]  Beth Erickson,et al.  Image Guided Cervical Brachytherapy: 2014 Survey of the American Brachytherapy Society. , 2016, International journal of radiation oncology, biology, physics.

[7]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[8]  Christian Kirisits,et al.  Gynecologic radiation therapy: Novel approaches to image-guidance and management , 2011 .

[9]  Elena De Momi,et al.  Blood vessel segmentation algorithms - Review of methods, datasets and evaluation metrics , 2018, Comput. Methods Programs Biomed..

[10]  Krzysztof Krawiec,et al.  Segmenting Retinal Blood Vessels With Deep Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[11]  Xuenan Cui,et al.  Deep CT to MR Synthesis Using Paired and Unpaired Data , 2018, Sensors.

[12]  Jan Egger,et al.  3T MR-Guided Brachytherapy for Gynecologic Malignancies , 2012, Magnetic resonance imaging.

[13]  Max A. Viergever,et al.  Multiscale vessel tracking , 2004, IEEE Transactions on Medical Imaging.

[14]  J. Dimopoulos,et al.  The Vienna applicator for combined intracavitary and interstitial brachytherapy of cervical cancer: clinical feasibility and preliminary results. , 2006, International journal of radiation oncology, biology, physics.

[15]  Christian Kirisits,et al.  Comprar Gynecologic Radiation Therapy. Novel Approaches To Image-Guidance And Management | A. N. Viswanathan | 9783540689546 | Springer , 2010 .

[16]  Robert A Cormack,et al.  Comparison of outcomes for MR-guided versus CT-guided high-dose-rate interstitial brachytherapy in women with locally advanced carcinoma of the cervix. , 2017, Gynecologic oncology.

[17]  Christian Cachard,et al.  Model Fitting Using RANSAC for Surgical Tool Localization in 3-D Ultrasound Images , 2010, IEEE Transactions on Biomedical Engineering.

[18]  Eugene Wong,et al.  Simultaneous automatic segmentation of multiple needles using 3D ultrasound for high‐dose‐rate prostate brachytherapy , 2017, Medical physics.

[19]  Tanweer Rashid,et al.  MRI-Based Medial Axis Extraction and Boundary Segmentation of Cranial Nerves Through Discrete Deformable 3D Contour and Surface Models , 2017, IEEE Transactions on Medical Imaging.

[20]  Phillip M Devlin,et al.  Magnetic resonance-guided interstitial therapy for vaginal recurrence of endometrial cancer. , 2006, International journal of radiation oncology, biology, physics.

[21]  Robert D. Howe,et al.  GPU based real-time instrument tracking with three-dimensional ultrasound , 2007, Medical Image Anal..

[22]  Max A. Viergever,et al.  ConvNet-Based Localization of Anatomical Structures in 3-D Medical Images , 2017, IEEE Transactions on Medical Imaging.

[23]  Luis Ibáñez,et al.  The Design of SimpleITK , 2013, Front. Neuroinform..

[24]  Jagadeesan Jayender,et al.  Multimodal imaging for improved diagnosis and treatment of cancers , 2015, Cancer.

[25]  Kemal Tuncali,et al.  Automatic Needle Segmentation and Localization in MRI With 3-D Convolutional Neural Networks: Application to MRI-Targeted Prostate Biopsy , 2019, IEEE Transactions on Medical Imaging.

[26]  Purang Abolmaesumi,et al.  Automatic Quality Assessment of Echocardiograms Using Convolutional Neural Networks: Feasibility on the Apical Four-Chamber View , 2017, IEEE Transactions on Medical Imaging.

[27]  Sabee Molloi,et al.  Detecting Cardiovascular Disease from Mammograms With Deep Learning , 2017, IEEE Transactions on Medical Imaging.

[28]  Paolo Zaffino,et al.  Technical Note: plastimatch mabs, an open source tool for automatic image segmentation. , 2016, Medical physics.

[29]  John Salvatier,et al.  Theano: A Python framework for fast computation of mathematical expressions , 2016, ArXiv.

[30]  Milan Sonka,et al.  3D Slicer as an image computing platform for the Quantitative Imaging Network. , 2012, Magnetic resonance imaging.

[31]  Carl-Fredrik Westin,et al.  Automatic Tractography Segmentation Using a High-Dimensional White Matter Atlas , 2007, IEEE Transactions on Medical Imaging.

[32]  Jochen Franke,et al.  Intraoperative detection and localization of cylindrical implants in cone-beam CT image data , 2014, International Journal of Computer Assisted Radiology and Surgery.

[33]  Heinz Handels,et al.  Efficient patient modeling for visuo-haptic VR simulation using a generic patient atlas , 2016, Comput. Methods Programs Biomed..

[34]  Touradj Ebrahimi,et al.  MESH: measuring errors between surfaces using the Hausdorff distance , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[35]  Jackie Szymonifka,et al.  A prospective trial of real-time magnetic resonance-guided catheter placement in interstitial gynecologic brachytherapy. , 2013, Brachytherapy.

[36]  Elena Marchiori,et al.  Location Sensitive Deep Convolutional Neural Networks for Segmentation of White Matter Hyperintensities , 2016, Scientific Reports.

[37]  Marc Morcos,et al.  Interventional Radiation Oncology (IRO): Transition of a magnetic resonance simulator to a brachytherapy suite. , 2018, Brachytherapy.

[38]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[39]  Wei Wang,et al.  Validation of Catheter Segmentation for MR-Guided Gynecologic Cancer Brachytherapy , 2013, MICCAI.

[40]  Gabor Fichtinger,et al.  A new scheme for curved needle segmentation in three-dimensional ultrasound images , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[41]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[42]  Purang Abolmaesumi,et al.  Correction to “Automatic Quality Assessment of Echocardiograms Using Convolutional Neural Networks: Feasibility on the Apical Four-Chamber View” , 2017, IEEE Transactions on Medical Imaging.

[43]  Purang Abolmaesumi,et al.  Classification of clinical significance of MRI prostate findings using 3D convolutional neural networks , 2017, Medical Imaging.

[44]  J. Dimopoulos,et al.  Recommendations from gynaecological (GYN) GEC ESTRO working group (II): concepts and terms in 3D image-based treatment planning in cervix cancer brachytherapy-3D dose volume parameters and aspects of 3D image-based anatomy, radiation physics, radiobiology. , 2006, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[45]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[46]  G Baroni,et al.  Validation of Automatic Contour Propagation for 4D Treatment Planning Using Multiple Metrics , 2013, Technology in cancer research & treatment.

[47]  Pierangela Bruno,et al.  Using CNNs for Designing and Implementing an Automatic Vascular Segmentation Method of Biomedical Images , 2018, AI*IA.

[48]  Robert Rohling,et al.  Needle Trajectory and Tip Localization in Real-Time 3-D Ultrasound Using a Moving Stylus. , 2015, Ultrasound in medicine & biology.

[49]  Céline Fouard,et al.  Segmentation, Separation and Pose Estimation of Prostate Brachytherapy Seeds in CT Images , 2015, IEEE Transactions on Biomedical Engineering.

[50]  Ruibin Ma,et al.  Accurate model‐based segmentation of gynecologic brachytherapy catheter collections in MRI‐images , 2017, Medical Image Anal..

[51]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.