Uncontrolled Confounders May Lead to False or Overvalued Radiomics Signature: A Proof of Concept Using Survival Analysis in a Multicenter Cohort of Kidney Cancer

Purpose We aimed to explore potential confounders of prognostic radiomics signature predicting survival outcomes in clear cell renal cell carcinoma (ccRCC) patients and demonstrate how to control for them. Materials and Methods Preoperative contrast enhanced abdominal CT scan of ccRCC patients along with pathological grade/stage, gene mutation status, and survival outcomes were retrieved from The Cancer Imaging Archive (TCIA)/The Cancer Genome Atlas—Kidney Renal Clear Cell Carcinoma (TCGA-KIRC) database, a publicly available dataset. A semi-automatic segmentation method was applied to segment ccRCC tumors, and 1,160 radiomics features were extracted from each segmented tumor on the CT images. Non-parametric principal component decomposition (PCD) and unsupervised hierarchical clustering were applied to build the radiomics signature models. The factors confounding the radiomics signature were investigated and controlled sequentially. Kaplan–Meier curves and Cox regression analyses were performed to test the association between radiomics signatures and survival outcomes. Results 183 patients of TCGA-KIRC cohort with available imaging, pathological, and clinical outcomes were included in this study. All 1,160 radiomics features were included in the first radiomics signature. Three additional radiomics signatures were then modelled in successive steps removing redundant radiomics features first, removing radiomics features biased by CT slice thickness second, and removing radiomics features dependent on tumor size third. The final radiomics signature model was the most parsimonious, unbiased by CT slice thickness, and independent of tumor size. This final radiomics signature stratified the cohort into radiomics phenotypes that are different by cancer-specific and recurrence-free survival; HR (95% CI) = 3.0 (1.5–5.7), p <0.05 and HR (95% CI) = 6.6 (3.1–14.1), p <0.05, respectively. Conclusion Radiomics signature can be confounded by multiple factors, including feature redundancy, image acquisition parameters like slice thickness, and tumor size. Attention to and proper control for these potential confounders are necessary for a reliable and clinically valuable radiomics signature.

[1]  Habib Zaidi,et al.  Radiomics-based machine learning model to predict risk of death within 5-years in clear cell renal cell carcinoma patients , 2020, Comput. Biol. Medicine.

[2]  Dimitris Visvikis,et al.  Harmonization strategies for multicenter radiomics investigations , 2020, Physics in medicine and biology.

[3]  D. Dong,et al.  Predicting response to immunotherapy in advanced non-small-cell lung cancer using tumor mutational burden radiomic biomarker , 2020, Journal for ImmunoTherapy of Cancer.

[4]  Lawrence H. Schwartz,et al.  Radiomics Prediction of EGFR Status in Lung Cancer—Our Experience in Using Multiple Feature Extractors and The Cancer Imaging Archive Data , 2020, Tomography.

[5]  Lin Lu,et al.  Identification of Non–Small Cell Lung Cancer Sensitive to Systemic Cancer Therapies Using Radiomics , 2020, Clinical Cancer Research.

[6]  R. Steenbakkers,et al.  The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping. , 2020, Radiology.

[7]  L. Schwartz,et al.  A quantitative imaging biomarker for predicting disease-free-survival-associated histologic subgroups in lung adenocarcinoma , 2020, European Radiology.

[8]  E. Sala,et al.  Radiomics of computed tomography and magnetic resonance imaging in renal cell carcinoma—a systematic review and meta-analysis , 2020, European Radiology.

[9]  Lin Lu,et al.  Radiomics Response Signature for Identification of Metastatic Colorectal Cancer Sensitive to Therapies Targeting EGFR Pathway. , 2020, Journal of the National Cancer Institute.

[10]  H. Zaidi,et al.  Radiomics for classification of bone mineral loss: A machine learning study. , 2020, Diagnostic and interventional imaging.

[11]  Binsheng Zhao,et al.  Automatic Liver Segmentation by Integrating Fully Convolutional Networks into Active Contour Models. , 2019, Medical physics.

[12]  M. Deevband,et al.  Noninvasive Fuhrman grading of clear cell renal cell carcinoma using computed tomography radiomic features and machine learning , 2019, La radiologia medica.

[13]  H. Abdollahi,et al.  Noninvasive O6 Methylguanine-DNA Methyltransferase Status Prediction in Glioblastoma Multiforme Cancer Using Magnetic Resonance Imaging Radiomics Features: Univariate and Multivariate Radiogenomics Analysis , 2019 .

[14]  Xin Zhen,et al.  Radiomics of small renal masses on multiphasic CT: accuracy of machine learning–based classification models for the differentiation of renal cell carcinoma and angiomyolipoma without visible fat , 2019, European Radiology.

[15]  Hassan Maleki,et al.  Next Generation Radiogenomics Sequencing for Prediction of EGFR and KRAS Mutation Status in NSCLC Patients Using Multimodal Imaging and Machine Learning Approaches , 2019, ArXiv.

[16]  D. Rubin,et al.  Radiomics and Radiogenomics , 2019, Machine and Deep Learning in Oncology, Medical Physics and Radiology.

[17]  L. Schwartz,et al.  Nonenhancing Component of Clear Cell Renal Cell Carcinoma on Computed Tomography Correlates With Tumor Necrosis and Stage and Serves as a Size-Independent Prognostic Biomarker. , 2019, Journal of computer assisted tomography.

[18]  Fanny Orlhac,et al.  Validation of A Method to Compensate Multicenter Effects Affecting CT Radiomics. , 2019, Radiology.

[19]  Benjamin Haibe-Kains,et al.  Vulnerabilities of radiomic signature development: The need for safeguards. , 2019, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[20]  L. Schwartz,et al.  CT Slice Thickness and Convolution Kernel Affect Performance of a Radiomic Model for Predicting EGFR Status in Non-Small Cell Lung Cancer: A Preliminary Study , 2018, Scientific Reports.

[21]  N. Paragios,et al.  A radiomics approach to assess tumour-infiltrating CD8 cells and response to anti-PD-1 or anti-PD-L1 immunotherapy: an imaging biomarker, retrospective multicohort study. , 2018, The Lancet. Oncology.

[22]  L. Qiu,et al.  A preliminary study , 2018, Medicine.

[23]  J. Canales‐Vázquez,et al.  Radiomics of CT Features May Be Nonreproducible and Redundant: Influence of CT Acquisition Parameters. , 2018, Radiology.

[24]  M. Haider,et al.  Radiomics analysis at PET/CT contributes to prognosis of recurrence and survival in lung cancer treated with stereotactic body radiotherapy , 2018, Scientific Reports.

[25]  M. Hatt,et al.  Responsible Radiomics Research for Faster Clinical Translation , 2017, The Journal of Nuclear Medicine.

[26]  Andriy Fedorov,et al.  Computational Radiomics System to Decode the Radiographic Phenotype. , 2017, Cancer research.

[27]  Steffen Löck,et al.  A comparative study of machine learning methods for time-to-event survival data for radiomics risk modelling , 2017, Scientific Reports.

[28]  P. Lambin,et al.  Radiomics: the bridge between medical imaging and personalized medicine , 2017, Nature Reviews Clinical Oncology.

[29]  John Quackenbush,et al.  Somatic Mutations Drive Distinct Imaging Phenotypes in Lung Cancer. , 2017, Cancer research.

[30]  Lawrence H. Schwartz,et al.  Assessing Agreement between Radiomic Features Computed for Multiple CT Imaging Settings , 2016, PloS one.

[31]  Samuel H. Hawkins,et al.  Predicting Malignant Nodules from Screening CT Scans , 2016, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[32]  Zaiyi Liu,et al.  Effects of contrast-enhancement, reconstruction slice thickness and convolution kernel on the diagnostic performance of radiomics signature in solitary pulmonary nodule , 2016, Scientific Reports.

[33]  Erich P Huang,et al.  RECIST 1.1-Update and clarification: From the RECIST committee. , 2016, European journal of cancer.

[34]  W. Tsai,et al.  Reproducibility of radiomics for deciphering tumor phenotype with imaging , 2016, Scientific Reports.

[35]  P. Lambin,et al.  Radiomic Machine-Learning Classifiers for Prognostic Biomarkers of Head and Neck Cancer , 2015, Front. Oncol..

[36]  Paul Kinahan,et al.  Radiomics: Images Are More than Pictures, They Are Data , 2015, Radiology.

[37]  Jinzhong Yang,et al.  IBEX: an open infrastructure software platform to facilitate collaborative work in radiomics. , 2015, Medical physics.

[38]  Zoltan Szallasi,et al.  Systematic Evaluation of the Prognostic Impact and Intratumour Heterogeneity of Clear Cell Renal Cell Carcinoma Biomarkers , 2014, European urology.

[39]  P. Lambin,et al.  Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach , 2014, Nature Communications.

[40]  Stephen M. Moore,et al.  The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository , 2013, Journal of Digital Imaging.

[41]  C. Sander,et al.  Adverse Outcomes in Clear Cell Renal Cell Carcinoma with Mutations of 3p21 Epigenetic Regulators BAP1 and SETD2: A Report by MSKCC and the KIRC TCGA Research Network , 2013, Clinical Cancer Research.

[42]  Bernard Fertil,et al.  Shape and Texture Indexes Application to Cell nuclei Classification , 2013, Int. J. Pattern Recognit. Artif. Intell..

[43]  Patrick Granton,et al.  Radiomics: extracting more information from medical images using advanced feature analysis. , 2012, European journal of cancer.

[44]  P. Tang,et al.  Clinical and molecular prognostic factors in renal cell carcinoma: what we know so far. , 2011, Hematology/oncology clinics of North America.

[45]  M. Pencina,et al.  On the C‐statistics for evaluating overall adequacy of risk prediction procedures with censored survival data , 2011, Statistics in medicine.

[46]  H. Abdi,et al.  Principal component analysis , 2010 .

[47]  Wong-Ho Chow,et al.  Epidemiology and risk factors for kidney cancer , 2010, Nature Reviews Urology.

[48]  J. Wandtke,et al.  Comparing thin-section and thick-section CT of pericardial sinuses and recesses. , 2003, AJR. American journal of roentgenology.

[49]  Xiaoou Tang,et al.  Texture information in run-length matrices , 1998, IEEE Trans. Image Process..

[50]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[52]  S. Störkel,et al.  Morphological classification of renal cancer , 2004, World Journal of Urology.

[53]  A. Kakabadse,et al.  What We Know So Far , 2004 .

[54]  C. Compton,et al.  AJCC Cancer Staging Manual , 2002, Springer New York.