A Machine Learning Challenge for Prognostic Modelling in Head and Neck Cancer Using Multi-modal Data

Accurate prognosis for an individual patient is a key component of precision oncology. Recent advances in machine learning have enabled the development of models using a wider range of data, including imaging. Radiomics aims to extract quantitative predictive and prognostic biomarkers from routine medical imaging, but evidence for computed tomography radiomics for prognosis remains inconclusive. We have conducted an institutional machine learning challenge to develop an accurate model for overall survival prediction in head and neck cancer using clinical data etxracted from electronic medical records and pre-treatment radiological images, as well as to evaluate the true added benefit of radiomics for head and neck cancer prognosis. Using a large, retrospective dataset of 2,552 patients and a rigorous evaluation framework, we compared 12 different submissions using imaging and clinical data, separately or in combination. The winning approach used non-linear, multitask learning on clinical data and tumour volume, achieving high prognostic accuracy for 2-year and lifetime survival prediction and outperforming models relying on clinical data only, engineered radiomics and deep learning. Combining all submissions in an ensemble model resulted in improved accuracy, with the highest gain from a image-based deep learning model. Our results show the potential of machine learning and simple, informative prognostic factors in combination with large datasets as a tool to guide personalized cancer care.

[1]  D. Brachman,et al.  Molecular biology of head and neck cancer. , 1994, Seminars in oncology.

[2]  Olivier Gevaert,et al.  Deep learning with multimodal representation for pancancer prognosis prediction , 2019, bioRxiv.

[3]  Kathryn J Fowler,et al.  Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers-From the Radiology Editorial Board. , 2019, Radiology.

[4]  A R Feinstein,et al.  Computer-aided prognosis. II. Development of a prognostic algorithm. , 1971, Archives of internal medicine.

[5]  Todd R McNutt,et al.  Predicting acute radiation induced xerostomia in head and neck Cancer using MR and CT Radiomics of parotid and submandibular glands , 2019, Radiation Oncology.

[6]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[7]  R. Steenbakkers,et al.  The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping. , 2020, Radiology.

[8]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[9]  G. Corrado,et al.  End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography , 2019, Nature Medicine.

[10]  W. Westra,et al.  Molecular pathology of head and neck cancer: implications for diagnosis, prognosis, and treatment. , 2009, Annual review of pathology.

[11]  Leonard Wee,et al.  Machine learning helps identifying volume-confounding effects in radiomics. , 2020, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[12]  Leonard Wee,et al.  Artificial intelligence‐based clinical decision support in modern medical physics: Selection, acceptance, commissioning, and quality assurance , 2020, Medical physics.

[13]  Sameem Abdul Kareem,et al.  Oral cancer prognosis based on clinicopathologic and genomic markers using a hybrid of feature selection and machine learning methods , 2012, BMC Bioinformatics.

[14]  J. Seuntjens,et al.  Deep learning in head & neck cancer outcome prediction , 2019, Scientific Reports.

[15]  Jing Ning,et al.  Radiomics features of the primary tumor fail to improve prediction of overall survival in large cohorts of CT- and PET-imaged head and neck cancer patients , 2019, PloS one.

[16]  F. Harrell,et al.  Prognostic/Clinical Prediction Models: Multivariable Prognostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors , 2005 .

[17]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[18]  Jayashree Kalpathy-Cramer,et al.  Matched computed tomography segmentation and demographic data for oropharyngeal cancer radiomics challenges , 2017, Scientific Data.

[19]  Jeff Shrager,et al.  Rapid learning for precision oncology , 2014, Nature Reviews Clinical Oncology.

[20]  Laura M. Heiser,et al.  A community effort to assess and improve drug sensitivity prediction algorithms , 2014, Nature Biotechnology.

[21]  Chen Sun,et al.  Revisiting Unreasonable Effectiveness of Data in Deep Learning Era , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[22]  Wei Xu,et al.  Deintensification candidate subgroups in human papillomavirus-related oropharyngeal cancer according to minimal risk of distant metastasis. , 2013, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[23]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  P. Lambin,et al.  Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach , 2014, Nature Communications.

[25]  Benjamin Haibe-Kains,et al.  A fuzzy gene expression-based computational approach improves breast cancer prognostication , 2010, Genome Biology.

[26]  Jinzhong Yang,et al.  Preliminary investigation into sources of uncertainty in quantitative imaging features , 2015, Comput. Medical Imaging Graph..

[27]  C. Cernea,et al.  Tumor volume as an independent predictive factor of worse survival in patients with oral cavity squamous cell carcinoma , 2017, Head & neck.

[28]  J. E. van Timmeren,et al.  Tracking tumor biology with radiomics: A systematic review utilizing a radiomics quality score. , 2018, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[29]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[30]  Stephane Fotso,et al.  Deep Neural Networks for Survival Analysis Based on a Multi-Task Framework , 2018, ArXiv.

[31]  Anshu Ankolekar,et al.  From multisource data to clinical decision aids in radiation oncology: the need for a clinical data science community. , 2020, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[32]  Andriy Fedorov,et al.  Computational Radiomics System to Decode the Radiographic Phenotype. , 2017, Cancer research.

[33]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[34]  Joshua M. Stuart,et al.  The Cancer Genome Atlas Pan-Cancer analysis project , 2013, Nature Genetics.

[35]  J C Costello,et al.  Seeking the Wisdom of Crowds Through Challenge‐Based Competitions in Biomedical Research , 2013, Clinical pharmacology and therapeutics.

[36]  Geraint Rees,et al.  Clinically applicable deep learning for diagnosis and referral in retinal disease , 2018, Nature Medicine.

[37]  P. Blanchard,et al.  Treatment de‐escalation in HPV‐positive oropharyngeal carcinoma: Ongoing trials, critical issues and perspectives , 2015, International journal of cancer.

[38]  Clifton D Fuller,et al.  Radiomics in head and neck cancer: from exploration to application. , 2016, Translational cancer research.

[39]  Dimitris Visvikis,et al.  Characterization of PET/CT images using texture analysis: the past, the present… any future? , 2016, European Journal of Nuclear Medicine and Molecular Imaging.

[40]  Trevor Hastie,et al.  Transparency and reproducibility in artificial intelligence , 2020, Nature.

[41]  Jianhua Ma,et al.  Multi-Level Multi-Modality Fusion Radiomics: Application to PET and CT Imaging for Prognostication of Head and Neck Cancer , 2019, IEEE Journal of Biomedical and Health Informatics.

[42]  Paul Kinahan,et al.  Radiomics: Images Are More than Pictures, They Are Data , 2015, Radiology.

[43]  P. Lambin,et al.  A Deep Look Into the Future of Quantitative Imaging in Oncology: A Statement of Working Principles and Proposal for Change. , 2018, International journal of radiation oncology, biology, physics.

[44]  C. Fuller,et al.  Imaging-Genomic Study of Head and Neck Squamous Cell Carcinoma: Associations Between Radiomic Phenotypes and Genomic Mechanisms via Integration of The Cancer Genome Atlas and The Cancer Imaging Archive. , 2019, JCO clinical cancer informatics.

[45]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[46]  Johannes A. Langendijk,et al.  Delta-radiomics features during radiotherapy improve the prediction of late xerostomia , 2019, Scientific Reports.

[47]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[48]  Olivier Gevaert,et al.  Development and validation of radiomic signatures of head and neck squamous cell carcinoma molecular features and subtypes , 2019, EBioMedicine.

[49]  Benjamin Haibe-Kains,et al.  Vulnerabilities of radiomic signature development: The need for safeguards. , 2019, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[50]  Christopher Holmes,et al.  Improving the quality of machine learning in health applications and clinical research , 2020, Nature Machine Intelligence.

[51]  A. Leung,et al.  A Shallow Convolutional Neural Network Predicts Prognosis of Lung Cancer Patients in Multi-Institutional CT-Image Data , 2020, Nature Machine Intelligence.

[52]  Issam El-Naqa,et al.  Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer , 2017, Scientific Reports.

[53]  Takaya Saito,et al.  The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets , 2015, PloS one.