Reproducibility analysis of multi-institutional paired expert annotations and radiomic features of the Ivy Glioblastoma Atlas Project (Ivy GAP) dataset.

PURPOSE The availability of radiographic MRI scans for the Ivy Glioblastoma Atlas Project (Ivy GAP) has opened up opportunities for development of radiomic markers for prognostic/predictive applications in Glioblastoma (GBM). In this work, we address two critical challenges with regard to developing robust radiomic approaches: (1) the lack of availability of reliable segmentation labels for GBM tumor sub-compartments (i.e., enhancing tumor, non-enhancing tumor core, peritumoral edematous/infiltrated tissue), and (2) identifying \reproducible" radiomic features that are robust to segmentation variability across readers/sites. ACQUISITION AND VALIDATION METHODS From TCIA's Ivy GAP cohort, we obtained a paired set (n=31) of expert annotations approved by two board-certified neuro-radiologists at the Hospital of the University of Pennsylvania (UPenn) and at CaseWestern Reserve University (CWRU). For these studies, we performed a reproducibility study that assessed the variability in (a) segmentation labels, and (b) radiomic features, between these paired annotations. The radiomic variability was assessed on a comprehensive panel of 11,700 radiomic features including intensity, volumetric, morphologic, histogram-based, and textural parameters, extracted for each of the paired sets of annotations. Our results demonstrated (a) a high level of inter-rater agreement (median value of DICE ≥ 0:8 for all sub-compartments), and (b) ≈ 24% of the extracted radiomic features being highly correlated (based on Spearman's rank correlation coefficient) to annotation variations. These robust features largely belonged to morphology (describing shape characteristics), intensity (capturing intensity profile statistics), and COLLAGE (capturing heterogeneity in gradient orientations) feature families. DATA FORMAT AND USAGE NOTES We make publicly available on TCIA's Analysis Results Directory, the complete set of (a) multi-institutional expert annotations for the tumor sub-compartments, (b) 11,700 radiomic features, and (c) the associated reproducibility meta-analysis. POTENTIAL APPLICATIONS The annotations and the associated meta-data for Ivy GAP is released with the purpose of enabling researchers towards developing image-based biomarkers for prognostic/predictive applications in GBM.

[1]  A. Rahmim,et al.  Repeatability of Radiomic Features in Magnetic Resonance Imaging of Glioblastoma: Test-Retest and Image Registration Analyses. , 2020, Medical physics.

[2]  Christos Davatzikos,et al.  Overall survival prediction in glioblastoma patients using structural magnetic resonance imaging (MRI): advanced radiomic features may compensate for lack of advanced MRI modalities , 2020, Journal of medical imaging.

[3]  R. Steenbakkers,et al.  The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping. , 2020, Radiology.

[4]  Christos Davatzikos,et al.  Cancer Imaging Phenomics via CaPTk: Multi-Institutional Prediction of Progression-Free Survival and Pattern of Recurrence in Glioblastoma , 2020, JCO clinical cancer informatics.

[5]  C. Fuller,et al.  Stability analysis of CT radiomic features with respect to segmentation variation in oropharyngeal cancer , 2019, Clinical and translational radiation oncology.

[6]  Hajar Moradmand,et al.  Impact of image preprocessing methods on reproducibility of radiomic features in multimodal magnetic resonance imaging in glioblastoma , 2019, Journal of applied clinical medical physics.

[7]  Christos Davatzikos,et al.  Skull-Stripping of Glioblastoma MRI Scans Using 3D Deep Learning , 2019, BrainLes@MICCAI.

[8]  Shanshan Wang,et al.  Glioma Grading Prediction by Exploring Radiomics and Deep Learning Features , 2019, ISICDM.

[9]  Harini Veeraraghavan,et al.  Impact of image preprocessing on the scanner dependence of multi-parametric MRI radiomic features and covariate shift in multi-institutional glioblastoma datasets , 2019, Physics in medicine and biology.

[10]  Harini Veeraraghavan,et al.  Reliability of tumor segmentation in glioblastoma: impact on the robustness of MRI-radiomic features. , 2019, Medical physics.

[11]  Anant Madabhushi,et al.  Radiomics of the lesion habitat on pre-treatment MRI predicts response to chemo-radiation therapy in Glioblastoma , 2019, Medical Imaging.

[12]  A. M. Darweesh,et al.  Role of Multi-Slice Computed Tomography Perfusion in Evaluation of the Pulmonary Nodules , 2019, International Journal of Medical Imaging.

[13]  Anant Madabhushi,et al.  Radiomic features from pretreatment biparametric MRI predict prostate cancer biochemical recurrence: Preliminary findings , 2018, Journal of magnetic resonance imaging : JMRI.

[14]  et al.,et al.  Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge , 2018, ArXiv.

[15]  E. Schott,et al.  Polymorphisms in the Toll-like receptor 3 (TLR3) gene are associated with the natural course of hepatitis B virus infection in Caucasian population , 2018, Scientific Reports.

[16]  Christos Davatzikos,et al.  In vivo evaluation of EGFRvIII mutation in primary glioblastoma patients via complex multiparametric MRI signature , 2018, Neuro-oncology.

[17]  Harini Veeraraghavan,et al.  Technical Note: Extension of CERR for computational radiomics: A comprehensive MATLAB platform for reproducible radiomics research. , 2018, Medical physics.

[18]  Fuhui Long,et al.  An anatomic transcriptional atlas of human glioblastoma , 2018, Science.

[19]  Christos Davatzikos,et al.  Cancer imaging phenomics toolkit: quantitative imaging analytics for precision diagnostics and predictive modeling of clinical outcome , 2018, Journal of medical imaging.

[20]  Prateek Prasanna,et al.  Radiogenomic analysis of hypoxia pathway is predictive of overall survival in Glioblastoma , 2018, Scientific Reports.

[21]  Christos Davatzikos,et al.  Brain Cancer Imaging Phenomics Toolkit (brain-CaPTk): An Interactive Platform for Quantitative Analysis of Glioblastoma , 2017, BrainLes@MICCAI.

[22]  Christos Davatzikos,et al.  Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features , 2017, Scientific Data.

[23]  A. Madabhushi,et al.  Intratumoral and peritumoral radiomics for the pretreatment prediction of pathological complete response to neoadjuvant chemotherapy based on breast DCE-MRI , 2017, Breast Cancer Research.

[24]  F. Arnaud,et al.  From core referencing to data re-use: two French national initiatives to reinforce paleodata stewardship (National Cyber Core Repository and LTER France Retro-Observatory) , 2017 .

[25]  Christos Davatzikos,et al.  In Vivo Detection of EGFRvIII in Glioblastoma via Perfusion Magnetic Resonance Imaging Signature Consistent with Deep Peritumoral Infiltration: The ϕ-Index , 2017, Clinical Cancer Research.

[26]  A. Madabhushi,et al.  Radiomic features from the peritumoral brain parenchyma on treatment-naïve multi-parametric MR imaging predict long versus short-term survival in glioblastoma multiforme: Preliminary findings , 2017, European Radiology.

[27]  Paul A. Yushkevich,et al.  FAST AUTOMATIC SEGMENTATION OF HIPPOCAMPAL SUBFIELDS AND MEDIAL TEMPORAL LOBE SUBREGIONS IN 3 TESLA AND 7 TESLA T2-WEIGHTED MRI , 2016, Alzheimer's & Dementia.

[28]  Adrian Smith,et al.  UTILITY OF PMOD IMAGE QUANTIFICATION SOFTWARE FOR PROCESSING [11C]PIB AND [18F]FLUTEMETAMOL IMAGES FOR SUVR QUANTITATION ON THE CENTILOID SCALE , 2016, Alzheimer's & Dementia.

[29]  Terry K Koo,et al.  A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. , 2016, Journal Chiropractic Medicine.

[30]  G. Biros,et al.  Imaging Surrogates of Infiltration Obtained Via Multiparametric Imaging Pattern Analysis Predict Subsequent Location of Recurrence of Glioblastoma. , 2016, Neurosurgery.

[31]  Luke Macyszyn,et al.  Imaging patterns predict patient survival and molecular subtype in glioblastoma via machine learning techniques. , 2016, Neuro-oncology.

[32]  Bilwaj Gaonkar,et al.  GLISTRboost: Combining Multimodal MRI Segmentation, Registration, and Biophysical Tumor Growth Modeling with Gradient Boosting Machines for Glioma Segmentation , 2015, Brainles@MICCAI.

[33]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[34]  Jill S. Barnholtz-Sloan,et al.  CBTRUS Statistical Report: Primary Brain and Central Nervous System Tumors Diagnosed in the United States in 2008-2012 , 2015, Neuro-oncology.

[35]  W. Niessen,et al.  Quantification of Heterogeneity as a Biomarker in Tumor Imaging: A Systematic Review , 2014, PloS one.

[36]  Prateek Prasanna,et al.  Co-occurrence of Local Anisotropic Gradient Orientations (CoLlAGe): Distinguishing Tumor Confounders and Molecular Subtypes on MRI , 2014, MICCAI.

[37]  J. Barnholtz-Sloan,et al.  CBTRUS statistical report: primary brain and central nervous system tumors diagnosed in the United States in 2007-2011. , 2012, Neuro-oncology.

[38]  Kirby G. Vosburgh,et al.  3D Slicer: A Platform for Subject-Specific Image Analysis, Visualization, and Clinical Support , 2014 .

[39]  Stephen M. Moore,et al.  The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository , 2013, Journal of Digital Imaging.

[40]  Sven Haller,et al.  Do brain T2/FLAIR white matter hyperintensities correspond to myelin loss in normal aging? A radiologic-neuropathologic correlation study , 2013, Acta neuropathologica communications.

[41]  T. Cloughesy,et al.  Probabilistic Radiographic Atlas of Glioblastoma Phenotypes , 2013, American Journal of Neuroradiology.

[42]  Stefan Bauer,et al.  A Skull-Stripping Filter for ITK , 2013, The Insight Journal.

[43]  C. Jaffe.  Imaging and genomics: is there a synergy? , 2012, Radiology.

[44]  Ferenc A. Jolesz,et al.  Radiogenomic Mapping of Edema/Cellular Invasion MRI-Phenotypes in Glioblastoma Multiforme , 2011, PloS one.

[45]  R. Jeraj,et al.  Variability of textural features in FDG PET images due to different acquisition modes and reconstruction parameters , 2010, Acta oncologica.

[46]  Brian B. Avants,et al.  N4ITK: Improved N3 Bias Correction , 2010, IEEE Transactions on Medical Imaging.

[47]  Adolf Pfefferbaum,et al.  The SRI24 multichannel atlas of normal adult human brain structure , 2009, Human brain mapping.

[48]  Guido Gerig,et al.  User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability , 2006, NeuroImage.

[49]  Xiaoou Tang,et al.  Texture information in run-length matrices , 1998, IEEE Trans. Image Process..

[50]  Belur V. Dasarathy,et al.  Image characterizations based on joint gray level-run length distributions , 1991, Pattern Recognit. Lett..

[51]  James F. Greenleaf,et al.  Use of gray value distribution of run lengths for texture analysis , 1990, Pattern Recognit. Lett..

[52]  J. Talairach,et al.  Co-Planar Stereotaxic Atlas of the Human Brain: 3-Dimensional Proportional System: An Approach to Cerebral Imaging , 1988 .

[53]  Robert King,et al.  Textural features corresponding to textural properties , 1989, IEEE Trans. Syst. Man Cybern..

[54]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[55]  Mary M. Galloway,et al.  Texture analysis using gray level run lengths , 1974 .

[56]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[57]  Joel Max,et al.  Quantizing for minimum distortion , 1960, IRE Trans. Inf. Theory.