Preoperative Brain Tumor Imaging: Models and Software for Segmentation and Standardized Reporting

For patients suffering from brain tumor, prognosis estimation and treatment decisions are made by a multidisciplinary team based on a set of preoperative MR scans. Currently, the lack of standardized and automatic methods for tumor detection and generation of clinical reports, incorporating a wide range of tumor characteristics, represents a major hurdle. In this study, we investigate the most occurring brain tumor types: glioblastomas, lower grade gliomas, meningiomas, and metastases, through four cohorts of up to 4,000 patients. Tumor segmentation models were trained using the AGU-Net architecture with different preprocessing steps and protocols. Segmentation performances were assessed in-depth using a wide-range of voxel and patient-wise metrics covering volume, distance, and probabilistic aspects. Finally, two software solutions have been developed, enabling an easy use of the trained models and standardized generation of clinical reports: Raidionics and Raidionics-Slicer. Segmentation performances were quite homogeneous across the four different brain tumor types, with an average true positive Dice ranging between 80 and 90%, patient-wise recall between 88 and 98%, and patient-wise precision around 95%. In conjunction to Dice, the identified most relevant other metrics were the relative absolute volume difference, the variation of information, and the Hausdorff, Mahalanobis, and object average symmetric surface distances. With our Raidionics software, running on a desktop computer with CPU support, tumor segmentation can be performed in 16–54 s depending on the dimensions of the MRI volume. For the generation of a standardized clinical report, including the tumor segmentation and features computation, 5–15 min are necessary. All trained models have been made open-access together with the source code for both software solutions and validation metrics computation. In the future, a method to convert results from a set of metrics into a final single score would be highly desirable for easier ranking across trained models. In addition, an automatic classification of the brain tumor type would be necessary to replace manual user input. Finally, the inclusion of post-operative segmentation in both software solutions will be key for generating complete post-operative standardized clinical reports.

[1]  Huan Minh Luu,et al.  Extending nn-UNet for brain tumor segmentation , 2021, BrainLes@MICCAI.

[2]  Mesut Melek,et al.  Roza: a new and comprehensive metric for evaluating classification systems , 2021, Computer methods in biomechanics and biomedical engineering.

[3]  M. Berger,et al.  Glioblastoma Surgery Imaging–Reporting and Data System: Validation and Performance of the Automated Segmentation Task , 2021, Cancers.

[4]  Christos Davatzikos,et al.  The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification , 2021, ArXiv.

[5]  G. Reifenberger,et al.  The 2021 WHO Classification of Tumors of the Central Nervous System: a summary. , 2021, Neuro-oncology.

[6]  M. Berger,et al.  Glioblastoma Surgery Imaging—Reporting and Data System: Standardized Reporting of Tumor Volume, Location, and Resectability Based on Automated Segmentations , 2021, Cancers.

[7]  Bjoern H Menze,et al.  Common Limitations of Image Processing Metrics: A Picture Story , 2021, ArXiv.

[8]  D. Rubin,et al.  Handling missing MRI sequences in deep learning segmentation of brain metastases: a multicenter study , 2021, npj Digital Medicine.

[9]  Ingerid Reinertsen,et al.  Meningioma Segmentation in T1-Weighted MRI Leveraging Global Context and Attention Mechanisms , 2021, Frontiers in Radiology.

[10]  Alexey Ershov,et al.  Introducing Biomedisa as an open-source online platform for biomedical image segmentation , 2020, Nature Communications.

[11]  Ingerid Reinertsen,et al.  Fast meningioma segmentation in T1-weighted magnetic resonance imaging volumes using a lightweight 3D deep learning architecture , 2020, Journal of medical imaging.

[12]  E. Neromyliotis,et al.  Machine Learning in Meningioma MRI: Past to Present. A Narrative Review , 2020, Journal of magnetic resonance imaging : JMRI.

[13]  B. Bartholmai,et al.  Implications of the updated Lung CT Screening Reporting and Data System (Lung-RADS version 1.1) for lung cancer screening , 2020, Journal of thoracic disease.

[14]  Millie Pant,et al.  Brain tumor segmentation and classification from magnetic resonance images: Review of selected methods from 2014 to 2019 , 2020, Pattern Recognit. Lett..

[15]  M. Perkuhn,et al.  Automated Meningioma Segmentation in Multiparametric MRI , 2020, Clinical Neuroradiology.

[16]  Antonio J. Plaza,et al.  Image Segmentation Using Deep Learning: A Survey , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Asif Mehmood,et al.  SoftAdapt: Techniques for Adaptive Loss Weighting of Neural Networks with Multi-Part Loss Functions , 2019, ArXiv.

[18]  Anne Line Stensjøen,et al.  Volumetric segmentation of glioblastoma progression compared to bidimensional products and clinical radiological reports , 2019, Acta Neurochirurgica.

[19]  Septimiu E. Salcudean,et al.  Reducing the Hausdorff Distance in Medical Image Segmentation With Convolutional Neural Networks , 2019, IEEE Transactions on Medical Imaging.

[20]  Daniel L. Rubin,et al.  Deep learning enables automatic detection and segmentation of brain metastases on multisequence MRI , 2019, Journal of magnetic resonance imaging : JMRI.

[21]  N. Butowski,et al.  Primary brain tumours in adults , 2018, The Lancet.

[22]  D. Maintz,et al.  Fully automated detection and segmentation of meningiomas using deep learning on routine multiparametric MRI , 2018, European Radiology.

[23]  J. Olson,et al.  Management-Based Structured Reporting of Posttreatment Glioma Response With the Brain Tumor Reporting and Data System. , 2018, Journal of the American College of Radiology : JACR.

[24]  Odelin Charron,et al.  Automatic detection and segmentation of brain metastases on multimodal MR images with a deep convolutional neural network , 2018, Comput. Biol. Medicine.

[25]  Steve B. Jiang,et al.  A deep convolutional neural network-based automatic delineation strategy for multiple brain metastases stereotactic radiosurgery , 2017, PloS one.

[26]  Christos Davatzikos,et al.  Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features , 2017, Scientific Data.

[27]  T. Huber,et al.  Reliability of Semi-Automated Segmentations in Glioblastoma , 2017, Clinical Neuroradiology.

[28]  Purang Abolmaesumi,et al.  DeepInfer: open-source deep learning deployment toolkit for image-guided therapy , 2017, Medical Imaging.

[29]  Konstantinos Kamnitsas,et al.  DeepMedic for Brain Tumor Segmentation , 2016, BrainLes@MICCAI.

[30]  Elisabetta Binaghi,et al.  Collection and fuzzy estimation of truth labels in glial tumour segmentation studies , 2016, Comput. methods Biomech. Biomed. Eng. Imaging Vis..

[31]  M. Götz,et al.  Radiomic Profiling of Glioblastoma: Identifying an Imaging Predictor of Patient Survival with Improved Performance over Established Clinical and Radiologic Risk Models. , 2016, Radiology.

[32]  G. Reifenberger,et al.  The 2016 World Health Organization Classification of Tumors of the Central Nervous System: a summary , 2016, Acta Neuropathologica.

[33]  J. Day,et al.  Neurocognitive Deficits and Neurocognitive Rehabilitation in Adult Brain Tumors , 2016, Current Treatment Options in Neurology.

[34]  Victor Alves,et al.  Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images , 2016, IEEE Transactions on Medical Imaging.

[35]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[36]  Marion Smits,et al.  Consensus recommendations for a standardized Brain Tumor Imaging Protocol in clinical trials. , 2015, Neuro-oncology.

[37]  Allan Hanbury,et al.  Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool , 2015, BMC Medical Imaging.

[38]  James V. Miller,et al.  GBM Volumetry using the 3D Slicer Medical Image Computing Platform , 2013, Scientific Reports.

[39]  E. Kasper,et al.  Extent of resection and radiotherapy in GBM: A 1973 to 2007 surveillance, epidemiology and end results analysis of 21,783 patients. , 2013, International journal of oncology.

[40]  Milan Sonka,et al.  3D Slicer as an image computing platform for the Quantitative Imaging Network. , 2012, Magnetic resonance imaging.

[41]  S. Lönn,et al.  Two-year survival of low-grade and high-grade glioma patients using data from the Swedish Cancer Registry , 2011, Acta Neurochirurgica.

[42]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[43]  Meritxell Bach Cuadra,et al.  A multidimensional segmentation evaluation for medical image data , 2009, Comput. Methods Programs Biomed..

[44]  C. Almli,et al.  Unbiased nonlinear average age-appropriate brain templates from birth to adulthood , 2009, NeuroImage.

[45]  Margaret Wrensch,et al.  Epidemiology of brain tumors. , 2007, Neurologic clinics.

[46]  Bernd W Scheithauer,et al.  The 2007 Revised World Health Organization (WHO) Classification of Tumours of the Central Nervous System: Newly Codified Entities , 2007, Brain pathology.

[47]  Carlo Tomasi,et al.  Image Similarity Using Mutual Information of Regions , 2004, ECCV.

[48]  Guido Gerig,et al.  Valmet: A New Validation Tool for Assessing and Improving 3D Object Segmentation , 2001, MICCAI.

[49]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[50]  Pierre Baldi,et al.  Assessing the accuracy of prediction algorithms for classification: an overview , 2000, Bioinform..

[51]  K R Hess,et al.  Neurosurgical outcomes in a modern series of 400 craniotomies for treatment of parenchymal tumors. , 1998, Neurosurgery.

[52]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[53]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[54]  Beth Sundheim,et al.  MUC-5 Evaluation Metrics , 1993, MUC.

[55]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[56]  L. R. Dice Measures of the Amount of Ecologic Association Between Species , 1945 .

[57]  P. Jaccard THE DISTRIBUTION OF THE FLORA IN THE ALPINE ZONE.1 , 1912 .

[58]  Klaus H. Maier-Hein,et al.  Abstract: nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation , 2019, Bildverarbeitung für die Medizin.

[59]  D. Margolis,et al.  PI-RADS Prostate Imaging - Reporting and Data System: 2015, Version 2. , 2016, European urology.

[60]  [World Health Organization classification of tumours of the central nervous system: a summary]. , 2016, Zhonghua bing li xue za zhi = Chinese journal of pathology.

[61]  Vladimir Vezhnevets,et al.  “GrowCut”-Interactive Multi-Label N-D Image Segmentation By Cellular Automata , 2005 .

[62]  Marina Meila,et al.  Comparing Clusterings by the Variation of Information , 2003, COLT.

[63]  L. Deangelis,et al.  Brain Tumors , 2019, Imaging Gliomas After Treatment.

[64]  G. McLachlan Mahalanobis distance , 1999 .

[65]  L. Hubert,et al.  Comparing partitions , 1985 .

[66]  P. Killeen,et al.  An Alternative to Null-Hypothesis Significance Tests , 2005, Psychological science.