Benchmarking Various Radiomic Toolkit Features While Applying the Image Biomarker Standardization Initiative toward Clinical Translation of Radiomic Analysis

The image biomarkers standardization initiative (IBSI) was formed to address the standardization of extraction of quantifiable imaging metrics. Despite its effort, there remains a lack of consensus or established guidelines regarding radiomic feature terminology, the underlying mathematics and their implementation across various software programs. This creates a scenario where features extracted using different toolboxes cannot be used to build or validate the same model leading to a non-generalization of radiomic results. In this study, IBSI-established phantom and benchmark values were used to compare the variation of the radiomic features while using 6 publicly available software programs and 1 in-house radiomics pipeline. All IBSI-standardized features (11 classes, 173 in total) were extracted. The relative differences between the extracted feature values from the different software programs and the IBSI benchmark values were calculated to measure the inter-software agreement. To better understand the variations, features are further grouped into 3 categories according to their properties: 1) morphology, 2) statistic/histogram and 3)texture features. While a good agreement was observed for a majority of radiomics features across the various tested programs, relatively poor agreement was observed for morphology features. Significant differences were also found in programs that use different gray-level discretization approaches. Since these software programs do not include all IBSI features, the level of quantitative assessment for each category was analyzed using Venn and UpSet diagrams and quantified using two ad hoc metrics. Morphology features earned lowest scores for both metrics, indicating that morphological features are not consistently evaluated among software programs. We conclude that radiomic features calculated using different software programs may not be interchangeable. Further studies are needed to standardize the workflow of radiomic feature extraction.

[1]  C. Halldin,et al.  Autoradiographic distribution of serotonin transporters and receptor subtypes in human brain , 2004, Human brain mapping.

[2]  L. Pace,et al.  Clinically significant prostate cancer detection on MRI: A radiomic shape features study. , 2019, European Journal of Radiology.

[3]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[4]  Thomas Lewiner,et al.  Efficient Implementation of Marching Cubes' Cases with Topological Guarantees , 2003, J. Graphics, GPU, & Game Tools.

[5]  P. Sathyanarayana,et al.  Image Texture Feature Extraction Using GLCM Approach , 2013 .

[6]  Raymond H Mak,et al.  Peritumoral radiomics features predict distant metastasis in locally advanced NSCLC , 2018, PloS one.

[7]  A. Rahmim,et al.  Machine Learning Methods for Optimal Radiomics-Based Differentiation Between Recurrence and Inflammation: Application to Nasopharyngeal Carcinoma Post-therapy PET/CT Images , 2019, Molecular Imaging and Biology.

[8]  M. Götz,et al.  Radiomic Profiling of Glioblastoma: Identifying an Imaging Predictor of Patient Survival with Improved Performance over Established Clinical and Radiologic Risk Models. , 2016, Radiology.

[9]  Jinzhong Yang,et al.  Measuring Computed Tomography Scanner Variability of Radiomics Features , 2015, Investigative radiology.

[10]  Ho Yun Lee,et al.  Clinical impact of variability on CT radiomics and suggestions for suitable feature selection: a focus on lung cancer , 2019, Cancer Imaging.

[11]  Hong Yi,et al.  A survey of the marching cubes algorithm , 2006, Comput. Graph..

[12]  Benjamin Haibe-Kains,et al.  Radiomic feature clusters and Prognostic Signatures specific for Lung and Head & Neck cancer , 2015, Scientific Reports.

[13]  Ron Kikinis,et al.  Repeatability of Multiparametric Prostate MRI Radiomics Features , 2018, Scientific Reports.

[14]  Jesús Angulo,et al.  Advanced Statistical Matrices for Texture Characterization: Application to Cell Classification , 2014, IEEE Transactions on Biomedical Engineering.

[15]  Daniel L. Rubin,et al.  Quantitative Image Feature Engine (QIFE): an Open-Source, Modular Engine for 3D Quantitative Feature Extraction from Volumetric Medical Images , 2018, Journal of Digital Imaging.

[16]  S. Ashrafinia.  QUANTITATIVE NUCLEAR MEDICINE IMAGING USING ADVANCED IMAGE RECONSTRUCTION AND RADIOMICS , 2019 .

[17]  P. Lambin,et al.  Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach , 2014, Nature Communications.

[18]  Jacob D. Furst,et al.  RUN-LENGTH ENCODING FOR VOLUMETRIC TEXTURE , 2004 .

[19]  P. Lambin,et al.  Exploratory Study to Identify Radiomics Classifiers for Lung Cancer Histology , 2016, Front. Oncol..

[20]  Akshada A. Gade,et al.  Feature Extraction using GLCM for Dietary Assessment Application , 2018 .

[21]  Bernard Fertil,et al.  Shape and Texture Indexes Application to Cell nuclei Classification , 2013, Int. J. Pattern Recognit. Artif. Intell..

[22]  Martin Sill,et al.  Large-scale Radiomic Profiling of Recurrent Glioblastoma Identifies an Imaging Predictor for Stratifying Anti-Angiogenic Treatment Response , 2016, Clinical Cancer Research.

[23]  James F. Greenleaf,et al.  Use of gray value distribution of run lengths for texture analysis , 1990, Pattern Recognit. Lett..

[24]  William G. Wee,et al.  Neighboring gray level dependence matrix for texture classification , 1982, Comput. Graph. Image Process..

[25]  H. Aerts,et al.  Radiomics performs comparable to morphologic assessment by expert radiologists for prediction of response to neoadjuvant chemoradiotherapy on baseline staging MRI in rectal cancer , 2019, Abdominal Radiology.

[26]  O. Riesterer,et al.  Stability of radiomic features in CT perfusion maps , 2016, Physics in medicine and biology.

[27]  Luke Macyszyn,et al.  Imaging patterns predict patient survival and molecular subtype in glioblastoma via machine learning techniques. , 2016, Neuro-oncology.

[28]  Robert King,et al.  Textural features corresponding to textural properties , 1989, IEEE Trans. Syst. Man Cybern..

[29]  Ronald Boellaard,et al.  Repeatability of metabolically active tumor volume measurements with FDG PET/CT in advanced gastrointestinal malignancies: a multicenter study. , 2014, Radiology.

[30]  Issam El-Naqa,et al.  Exploring feature-based approaches in PET images for predicting cancer treatment outcomes , 2009, Pattern Recognit..

[31]  Maria Antonietta Mazzei,et al.  Texture analysis as a predictor of radiation-induced xerostomia in head and neck patients undergoing IMRT , 2018, La radiologia medica.

[32]  Christos Davatzikos,et al.  Radiomic signature of infiltration in peritumoral edema predicts subsequent recurrence in glioblastoma: implications for personalized radiotherapy planning , 2018, Journal of medical imaging.

[33]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[34]  Beatriz Paniagua,et al.  Software comparison to analyze bone radiomics from high resolution CBCT scans of mandibular condyles. , 2019, Dento maxillo facial radiology.

[35]  A. Rahmim,et al.  Application of Texture and Radiomics Analysis to Clinical Myocardial Perfusion SPECT Imaging , 2018 .

[36]  Issam El-Naqa,et al.  Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer , 2017, Scientific Reports.

[37]  Mary M. Galloway,et al.  Texture analysis using gray level run lengths , 1974 .

[38]  Marco Nolden,et al.  MITK Phenotyping: An open-source toolchain for image-based personalized medicine with radiomics. , 2019, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[39]  Anne Humeau-Heurtier,et al.  Texture Feature Extraction Methods: A Survey , 2019, IEEE Access.

[40]  Jing Xu,et al.  A Computer-Assisted Proof of Correctness of a Marching Cubes Algorithm , 2013, IMR.

[41]  El Naqa,et al.  A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities , 2015 .

[42]  R. Wahl,et al.  From RECIST to PERCIST: Evolving Considerations for PET Response Criteria in Solid Tumors , 2009, Journal of Nuclear Medicine.

[43]  Run-Length Matrices For Texture Analysis , 2011, The Insight Journal.

[44]  Hania Al-Hallaq,et al.  Variation in algorithm implementation across radiomics software , 2018, Journal of medical imaging.

[45]  Jinzhong Yang,et al.  Computational resources for radiomics , 2016 .

[46]  R. Steenbakkers,et al.  The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping. , 2020, Radiology.

[47]  W E Bolch,et al.  Marching cube algorithm: review and trilinear interpolation adaptation for image-based dosimetric models. , 2003, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[48]  Michael Unser,et al.  Sum and Difference Histograms for Texture Classification , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Andriy Fedorov,et al.  Computational Radiomics System to Decode the Radiographic Phenotype. , 2017, Cancer research.

[50]  Andre Dekker,et al.  Radiomics: the process and the challenges. , 2012, Magnetic resonance imaging.

[51]  Christos Davatzikos,et al.  Cancer imaging phenomics toolkit: quantitative imaging analytics for precision diagnostics and predictive modeling of clinical outcome , 2018, Journal of medical imaging.

[52]  Bernard Fertil,et al.  Texture indexes and gray level size zone matrix. Application to cell nuclei classification , 2009 .

[53]  Geoffrey G. Zhang,et al.  Intrinsic dependencies of CT radiomic features on voxel size and number of gray levels , 2017, Medical physics.

[54]  K S Nikita,et al.  A novel and efficient implementation of the marching cubes algorithm. , 2001, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[55]  M. Giger,et al.  Deciphering Genomic Underpinnings of Quantitative MRI-based Radiomic Phenotypes of Invasive Breast Carcinoma , 2015, Scientific Reports.

[56]  Xiaoou Tang,et al.  Texture information in run-length matrices , 1998, IEEE Trans. Image Process..

[57]  Irène Buvat,et al.  LIFEx: A Freeware for Radiomic Feature Calculation in Multimodality Imaging to Accelerate Advances in the Characterization of Tumor Heterogeneity. , 2018, Cancer research.

[58]  Ronald Boellaard,et al.  Repeatability of Radiomic Features in Non-Small-Cell Lung Cancer [18F]FDG-PET/CT Studies: Impact of Reconstruction and Delineation , 2016, Molecular Imaging and Biology.

[59]  P. Lambin,et al.  CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma. , 2015, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[60]  Christos Davatzikos,et al.  The Cancer Imaging Phenomics Toolkit (CaPTk): Technical Overview , 2019, BrainLes@MICCAI.