A comparative study of machine learning methods for time-to-event survival data for radiomics risk modelling

Radiomics applies machine learning algorithms to quantitative imaging data to characterise the tumour phenotype and predict clinical outcome. For the development of radiomics risk models, a variety of different algorithms is available and it is not clear which one gives optimal results. Therefore, we assessed the performance of 11 machine learning algorithms combined with 12 feature selection methods by the concordance index (C-Index), to predict loco-regional tumour control (LRC) and overall survival for patients with head and neck squamous cell carcinoma. The considered algorithms are able to deal with continuous time-to-event survival data. Feature selection and model building were performed on a multicentre cohort (213 patients) and validated using an independent cohort (80 patients). We found several combinations of machine learning algorithms and feature selection methods which achieve similar results, e.g., MSR-RF: C-Index = 0.71 and BT-COX: C-Index = 0.70 in combination with Spearman feature selection. Using the best performing models, patients were stratified into groups of low and high risk of recurrence. Significant differences in LRC were obtained between both groups on the validation cohort. Based on the presented analysis, we identified a subset of algorithms which should be considered in future radiomics studies to develop stable and clinically relevant predictive models for time-to-event endpoints.

[1]  Mechthild Krause,et al.  Radiation oncology in the era of precision medicine , 2016, Nature Reviews Cancer.

[2]  P. Lambin,et al.  Machine Learning methods for Quantitative Radiomic Biomarkers , 2015, Scientific Reports.

[3]  William G. Wee,et al.  Neighboring gray level dependence matrix for texture classification , 1982, Comput. Graph. Image Process..

[4]  P. Lambin,et al.  Radiomic Machine-Learning Classifiers for Prognostic Biomarkers of Head and Neck Cancer , 2015, Front. Oncol..

[5]  M. Pencina,et al.  Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation , 2004, Statistics in medicine.

[6]  Sang Joon Park,et al.  Impact of Reconstruction Algorithms on CT Radiomic Features of Pulmonary Tumors: Analysis of Intra- and Inter-Reader Variability and Inter-Reconstruction Algorithm Variability , 2016, PloS one.

[7]  Samuel H. Hawkins,et al.  Deep Feature Transfer Learning in Combination with Traditional Features Predicts Survival Among Patients with Lung Adenocarcinoma , 2016, Tomography.

[8]  Ronald M. Summers,et al.  Improving Computer-Aided Detection Using Convolutional Neural Networks and Random View Aggregation , 2015, IEEE Transactions on Medical Imaging.

[9]  H. Aerts,et al.  Applications and limitations of radiomics , 2016, Physics in medicine and biology.

[10]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[11]  Jesús Angulo,et al.  Advanced Statistical Matrices for Texture Characterization: Application to Cell Classification , 2014, IEEE Transactions on Biomedical Engineering.

[12]  Di Dong,et al.  Non-small cell lung cancer: quantitative phenotypic analysis of CT images as a potential marker of prognosis , 2016, Scientific Reports.

[13]  Wagner Coelho A. Pereira,et al.  Analysis of Co-Occurrence Texture Statistics as a Function of Gray-Level Quantization for Classifying Breast Ultrasound , 2012, IEEE Transactions on Medical Imaging.

[14]  Robert King,et al.  Textural features corresponding to textural properties , 1989, IEEE Trans. Syst. Man Cybern..

[15]  Benjamin Haibe-Kains,et al.  Radiomic feature clusters and Prognostic Signatures specific for Lung and Head & Neck cancer , 2015, Scientific Reports.

[16]  P. Lambin,et al.  Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach , 2014, Nature Communications.

[17]  D. Aust,et al.  HPV status, cancer stem cell marker expression, hypoxia gene signatures and tumour volume identify good prognosis subgroups in patients with HNSCC after primary radiochemotherapy: A multicentre retrospective study of the German Cancer Consortium Radiation Oncology Group (DKTK-ROG). , 2016, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[18]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[19]  Michael Baumann,et al.  Exploratory prospective trial of hypoxia-specific PET imaging during radiochemotherapy in patients with locally advanced head-and-neck cancer. , 2012, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[20]  Bernard Fertil,et al.  Texture indexes and gray level size zone matrix. Application to cell nuclei classification , 2009 .

[21]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[22]  Ronald M. Summers,et al.  Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique , 2016 .

[23]  F. Harrell,et al.  Prognostic/Clinical Prediction Models: Multivariable Prognostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors , 2005 .

[24]  El Naqa,et al.  A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities , 2015 .

[25]  W. Tsai,et al.  Reproducibility of radiomics for deciphering tumor phenotype with imaging , 2016, Scientific Reports.

[26]  Victor Alves,et al.  Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images , 2016, IEEE Transactions on Medical Imaging.

[27]  Lawrence O. Hall,et al.  Exploring deep features from brain tumor magnetic resonance images via transfer learning , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[28]  Belur V. Dasarathy,et al.  Image characterizations based on joint gray level-run length distributions , 1991, Pattern Recognit. Lett..

[29]  A. van der Schaaf,et al.  CT image biomarkers to improve patient-specific prediction of radiation-induced xerostomia and sticky saliva. , 2017, Radiotherapy and Oncology.

[30]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[31]  Daniel B. Mark,et al.  TUTORIAL IN BIOSTATISTICS MULTIVARIABLE PROGNOSTIC MODELS: ISSUES IN DEVELOPING MODELS, EVALUATING ASSUMPTIONS AND ADEQUACY, AND MEASURING AND REDUCING ERRORS , 1996 .

[32]  Mary M. Galloway,et al.  Texture analysis using gray level run lengths , 1974 .

[33]  M. Götz,et al.  Radiomic Profiling of Glioblastoma: Identifying an Imaging Predictor of Patient Survival with Improved Performance over Established Clinical and Radiologic Risk Models. , 2016, Radiology.

[34]  David A Clausi An analysis of co-occurrence texture statistics as a function of grey level quantization , 2002 .

[35]  Steffen Löck,et al.  Image biomarker standardisation initiative - feature definitions , 2016, ArXiv.

[36]  Taghi M. Khoshgoftaar,et al.  An extensive comparison of feature ranking aggregation techniques in bioinformatics , 2012, 2012 IEEE 13th International Conference on Information Reuse & Integration (IRI).

[37]  Dimitris Visvikis,et al.  Characterization of PET/CT images using texture analysis: the past, the present… any future? , 2016, European Journal of Nuclear Medicine and Molecular Imaging.

[38]  J. Kekäläinen,et al.  Lectin staining and flow cytometry reveals female-induced sperm acrosome reaction and surface carbohydrate reorganization , 2015, Scientific Reports.

[39]  P. Lambin,et al.  CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma. , 2015, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[40]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[41]  Peter Dalgaard,et al.  R Development Core Team (2010): R: A language and environment for statistical computing , 2010 .