Machine and deep learning methods for radiomics.

Radiomics is an emerging area in quantitative image analysis that aims to relate large-scale extracted imaging information to clinical and biological endpoints. The development of quantitative imaging methods along with machine learning has enabled the opportunity to move data science research towards translation for more personalized cancer treatments. Accumulating evidence has indeed demonstrated that noninvasive advanced imaging analytics, that is, radiomics, can reveal key components of tumor phenotype for multiple three-dimensional lesions at multiple time points over and beyond the course of treatment. These developments in the use of CT, PET, US, and MR imaging could augment patient stratification and prognostication buttressing emerging targeted therapeutic approaches. In recent years, deep learning architectures have demonstrated their tremendous potential for image segmentation, reconstruction, recognition, and classification. Many powerful open-source and commercial platforms are currently available to embark in new research areas of radiomics. Quantitative imaging research, however, is complex and key statistical principles should be followed to realize its full potential. The field of radiomics, in particular, requires a renewed focus on optimal study design/reporting practices and standardization of image acquisition, feature calculation, and rigorous statistical analysis for the field to move forward. In this article, the role of machine and deep learning as a major computational vehicle for advanced model building of radiomics-based signatures or classifiers, and diverse clinical applications, working principles, research opportunities, and available computational platforms for radiomics will be reviewed with examples drawn primarily from oncology. We also address issues related to common applications in medical physics, such as standardization, feature extraction, model building, and validation.

[1]  Bulat Ibragimov,et al.  Segmentation of organs‐at‐risks in head and neck CT images using convolutional neural networks , 2017, Medical physics.

[2]  Dakai Zhang,et al.  Use of texture analysis based on contrast‐enhanced MRI to predict treatment response to chemoradiotherapy in nasopharyngeal carcinoma , 2016, Journal of magnetic resonance imaging : JMRI.

[3]  N. Paragios,et al.  A radiomics approach to assess tumour-infiltrating CD8 cells and response to anti-PD-1 or anti-PD-L1 immunotherapy: an imaging biomarker, retrospective multicohort study. , 2018, The Lancet. Oncology.

[4]  P. Marsden,et al.  False Discovery Rates in PET and CT Studies with Texture Features: A Systematic Review , 2015, PloS one.

[5]  Jeffrey D Bradley,et al.  Pretreatment 18F-FDG PET Textural Features in Locally Advanced Non–Small Cell Lung Cancer: Secondary Analysis of ACRIN 6668/RTOG 0235 , 2016, The Journal of Nuclear Medicine.

[6]  Paul Kinahan,et al.  Radiomics: Images Are More than Pictures, They Are Data , 2015, Radiology.

[7]  P. Lambin,et al.  CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma. , 2015, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[8]  Binsheng Zhao,et al.  Evaluating variability in tumor measurements from same-day repeat CT scans of patients with non-small cell lung cancer. , 2009, Radiology.

[9]  Kenji Suzuki,et al.  Pixel-Based Machine Learning in Medical Imaging , 2012, Int. J. Biomed. Imaging.

[10]  Ganesh Rao,et al.  Associating spatial diversity features of radiologically defined tumor habitats with epidermal growth factor receptor driver status and 12-month survival in glioblastoma: methods and preliminary investigation , 2015, Journal of medical imaging.

[11]  Si Eun Lee,et al.  Radiomics of US texture features in differential diagnosis between triple-negative breast cancer and fibroadenoma , 2018, Scientific Reports.

[12]  Irène Buvat,et al.  LIFEx: A Freeware for Radiomic Feature Calculation in Multimodality Imaging to Accelerate Advances in the Characterization of Tumor Heterogeneity. , 2018, Cancer research.

[13]  R. Boellaard Standards for PET Image Acquisition and Quantitative Data Analysis , 2009, Journal of Nuclear Medicine.

[14]  Natalia Antropova,et al.  A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets , 2017, Medical physics.

[15]  Issam El-Naqa,et al.  Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer , 2017, Scientific Reports.

[16]  Philippe Lambin,et al.  Interchangeability of a Radiomic Signature Between Conventional and Weekly Cone Beam Computed Tomography Allowing Response Prediction in Non-Small Cell Lung Cancer , 2016 .

[17]  Geoffrey G. Zhang,et al.  Intrinsic dependencies of CT radiomic features on voxel size and number of gray levels , 2017, Medical physics.

[18]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[19]  Qianjin Feng,et al.  Robustness of Radiomic Features in [11C]Choline and [18F]FDG PET/CT Imaging of Nasopharyngeal Carcinoma: Impact of Segmentation and Discretization , 2016, Molecular Imaging and Biology.

[20]  K. Aldape,et al.  Identification of noninvasive imaging surrogates for brain tumor gene-expression modules , 2008, Proceedings of the National Academy of Sciences.

[21]  H. Aerts Semantics Features : Phenotype Quantification by a Radiologist ’ s Expert Eye , 2016 .

[22]  V. Goh,et al.  Predicting Response to Neoadjuvant Chemotherapy with PET Imaging Using Convolutional Neural Networks , 2015, PloS one.

[23]  John D. Lafferty,et al.  Semi-supervised learning using randomized mincuts , 2004, ICML.

[24]  Joseph O Deasy,et al.  Predicting hypoxia status using a combination of contrast-enhanced computed tomography and [18F]-Fluorodeoxyglucose positron emission tomography radiomics features. , 2017, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[25]  Alexander Wong,et al.  SISC: End-to-End Interpretable Discovery Radiomics-Driven Lung Cancer Prediction via Stacked Interpretable Sequencing Cells , 2019, IEEE Access.

[26]  Florent Tixier,et al.  The Impact of Optimal Respiratory Gating and Image Noise on Evaluation of Intratumor Heterogeneity on 18F-FDG PET Imaging of Lung Cancer , 2016, The Journal of Nuclear Medicine.

[27]  Balaji Ganeshan,et al.  Texture analysis of non-small cell lung cancer on unenhanced computed tomography: initial evidence for a relationship with tumour glucose metabolism and stage , 2010, Cancer imaging : the official publication of the International Cancer Imaging Society.

[28]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[29]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[30]  Olivier Gevaert,et al.  Core samples for radiomics features that are insensitive to tumor segmentation: method and pilot study using CT images of hepatocellular carcinoma , 2015, Journal of medical imaging.

[31]  Paul Aljabar,et al.  Clinical evaluation of atlas and deep learning based automatic contouring for lung cancer. , 2017, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[32]  T. Niu,et al.  Rectal Cancer: Assessment of Neoadjuvant Chemoradiation Outcome based on Radiomics of Multiparametric MRI , 2016, Clinical Cancer Research.

[33]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Ashirbani Saha,et al.  Deep learning for segmentation of brain tumors: Impact of cross‐institutional training and testing , 2018, Medical physics.

[35]  Andre Dekker,et al.  Radiomics: the process and the challenges. , 2012, Magnetic resonance imaging.

[36]  Brian O'Sullivan,et al.  Development and validation of a radiomic signature to predict HPV (p16) status from standard CT imaging: a multicenter study , 2018, The British journal of radiology.

[37]  Jinzhong Yang,et al.  IBEX: an open infrastructure software platform to facilitate collaborative work in radiomics. , 2015, Medical physics.

[38]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  P. Lambin,et al.  Radiomics: the bridge between medical imaging and personalized medicine , 2017, Nature Reviews Clinical Oncology.

[40]  Florent Tixier,et al.  Visual Versus Quantitative Assessment of Intratumor 18F-FDG PET Uptake Heterogeneity: Prognostic Value in Non–Small Cell Lung Cancer , 2014, The Journal of Nuclear Medicine.

[41]  Benjamin Haibe-Kains,et al.  Radiomic feature clusters and Prognostic Signatures specific for Lung and Head & Neck cancer , 2015, Scientific Reports.

[42]  M. Hatt,et al.  Responsible Radiomics Research for Faster Clinical Translation , 2017, The Journal of Nuclear Medicine.

[43]  P. Lambin,et al.  Exploratory Study to Identify Radiomics Classifiers for Lung Cancer Histology , 2016, Front. Oncol..

[44]  M. Giger,et al.  Deciphering Genomic Underpinnings of Quantitative MRI-based Radiomic Phenotypes of Invasive Breast Carcinoma , 2015, Scientific Reports.

[45]  Gary S Collins,et al.  Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement , 2015, BMC Medicine.

[46]  Hemant Ishwaran,et al.  Random Survival Forests , 2008, Wiley StatsRef: Statistics Reference Online.

[47]  Martial Hebert,et al.  Semi-Supervised Self-Training of Object Detection Models , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[48]  Lei Xing,et al.  Radiomics and radiogenomics for precision radiotherapy , 2018, Journal of radiation research.

[49]  B Julesz,et al.  Experiments in the visual perception of texture. , 1975, Scientific American.

[50]  Fang Liu,et al.  Shell feature: a new radiomics descriptor for predicting distant failure after radiotherapy in non-small cell lung cancer and cervix cancer , 2017, Physics in medicine and biology.

[51]  Juan J. Martinez,et al.  Evaluation of tumor-derived MRI-texture features for discrimination of molecular subtypes and prediction of 12-month survival status in glioblastoma. , 2015, Medical physics.

[52]  M. Pencina,et al.  Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation , 2004, Statistics in medicine.

[53]  P. Royston,et al.  External validation of a Cox prognostic model: principles and methods , 2013, BMC Medical Research Methodology.

[54]  Robert J. Gillies,et al.  The effect of SUV discretization in quantitative FDG-PET Radiomics: the need for standardized methodology in tumor texture analysis , 2015, Scientific Reports.

[55]  Cornelis H. Slump,et al.  Repeatability of 18F‐FDG PET radiomic features: A phantom study to explore sensitivity to image reconstruction settings, noise, and delineation method , 2018, Medical physics.

[56]  Patrick Granton,et al.  Radiomics: extracting more information from medical images using advanced feature analysis. , 2012, European journal of cancer.

[57]  Lorenzo L. Pesce,et al.  Noise injection for training artificial neural networks: a comparison with weight decay and early stopping. , 2009, Medical physics.

[58]  S. Armato,et al.  Lung texture in serial thoracic computed tomography scans: correlation of radiomics-based features with radiation therapy dose and radiation pneumonitis development. , 2015, International journal of radiation oncology, biology, physics.

[59]  Wolfgang Weber,et al.  Reliability of PET/CT Shape and Heterogeneity Features in Functional and Morphologic Components of Non–Small Cell Lung Cancer Tumors: A Repeatability Analysis in a Prospective Multicenter Cohort , 2016, The Journal of Nuclear Medicine.

[60]  Yohann Tschudi,et al.  Prostate cancer radiomics and the promise of radiogenomics. , 2016, Translational cancer research.

[61]  Bo Zhang,et al.  Discriminatively Boosted Image Clustering with Fully Convolutional Auto-Encoders , 2017, Pattern Recognit..

[62]  Roger Schaer,et al.  QuantImage: An Online Tool for High-Throughput 3D Radiomics Feature Extraction in PET-CT , 2017 .

[63]  Y. Liu,et al.  Radiomic Features Are Associated With EGFR Mutation Status in Lung Adenocarcinomas. , 2016, Clinical lung cancer.

[64]  Chaofeng Liang,et al.  A Fully-Automatic Multiparametric Radiomics Model: Towards Reproducible and Prognostic Imaging Signature for Prediction of Overall Survival in Glioblastoma Multiforme , 2017, Scientific Reports.

[65]  Paulo Mazzoncini de Azevedo Marques,et al.  Radiomics-based features for pattern recognition of lung cancer histopathology and metastases , 2018, Comput. Methods Programs Biomed..

[66]  D. Rubin,et al.  Automated tracking of quantitative assessments of tumor burden in clinical trials. , 2014, Translational oncology.

[67]  P. Lambin,et al.  Machine Learning methods for Quantitative Radiomic Biomarkers , 2015, Scientific Reports.

[68]  Michael Götz,et al.  Radiomic Machine Learning for Characterization of Prostate Lesions with MRI: Comparison to ADC Values. , 2018, Radiology.

[69]  Peter Balter,et al.  Can radiomics features be reproducibly measured from CBCT images for patients with non-small cell lung cancer? , 2015, Medical physics.

[70]  Hang Chang,et al.  Unsupervised Transfer Learning via Multi-Scale Convolutional Sparse Coding for Biomedical Applications , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[71]  Predicting clinically significant prostate cancer using DCE-MRI habitat descriptors , 2018, Oncotarget.

[72]  Insuk Sohn,et al.  Decoding Tumor Phenotypes for ALK, ROS1, and RET Fusions in Lung Adenocarcinoma Using a Radiomics Approach , 2015, Medicine.

[73]  Daniel L. Rubin,et al.  Quantitative Image Feature Engine (QIFE): an Open-Source, Modular Engine for 3D Quantitative Feature Extraction from Volumetric Medical Images , 2018, Journal of Digital Imaging.

[74]  Issam El-Naqa,et al.  Exploring feature-based approaches in PET images for predicting cancer treatment outcomes , 2009, Pattern Recognit..

[75]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[76]  Sabine Van Huffel,et al.  Support vector methods for survival analysis: a comparison between ranking and regression approaches , 2011, Artif. Intell. Medicine.

[77]  Ronald Boellaard,et al.  Repeatability of Radiomic Features in Non-Small-Cell Lung Cancer [18F]FDG-PET/CT Studies: Impact of Reconstruction and Delineation , 2016, Molecular Imaging and Biology.

[78]  Michal Strzelecki,et al.  MaZda - A software package for image texture analysis , 2009, Comput. Methods Programs Biomed..

[79]  Jing Li,et al.  A predictive model for distinguishing radiation necrosis from tumour progression after gamma knife radiosurgery based on radiomic features from MR images , 2018, European Radiology.

[80]  Jinzhong Yang,et al.  Preliminary investigation into sources of uncertainty in quantitative imaging features , 2015, Comput. Medical Imaging Graph..

[81]  Yongyi Yang,et al.  Computer-Aided Detection and Diagnosis of Breast Cancer With Mammography: Recent Advances , 2009, IEEE Transactions on Information Technology in Biomedicine.

[82]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[83]  Matthias Guckenberger,et al.  Computed Tomography Radiomics Predicts HPV Status and Local Tumor Control After Definitive Radiochemotherapy in Head and Neck Squamous Cell Carcinoma. , 2017, International journal of radiation oncology, biology, physics.

[84]  Bulat Ibragimov,et al.  Prostate cancer classification with multiparametric MRI transfer learning model , 2019, Medical physics.

[85]  Jianya Zhou,et al.  [Clinicopathological characteristics and imaging features of pulmonary adenocarcinoma with micropapillary pattern]. , 2014, Zhonghua zhong liu za zhi [Chinese journal of oncology].

[86]  D. Ikeda,et al.  Unsupervised Clustering of Quantitative Image Phenotypes Reveals Breast Cancer Subtypes with Distinct Prognoses and Molecular Pathways , 2017, Clinical Cancer Research.

[87]  Philippe Lambin,et al.  Is there a causal relationship between genetic changes and radiomics-based image features? An in vivo preclinical experiment with doxycycline inducible GADD34 tumor cells. , 2015, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[88]  F. Yin,et al.  Investigation of the support vector machine algorithm to predict lung radiation-induced pneumonitis. , 2007, Medical physics.

[89]  Philippe Lambin,et al.  Feature selection methodology for longitudinal cone-beam CT radiomics , 2017, Acta oncologica.

[90]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[91]  Cheng Li,et al.  Adjusting batch effects in microarray expression data using empirical Bayes methods. , 2007, Biostatistics.

[92]  Issam El Naqa,et al.  Development of a Fully Cross-Validated Bayesian Network Approach for Local Control Prediction in Lung Cancer , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.

[93]  I. El Naqa,et al.  Beyond imaging: The promise of radiomics. , 2017, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[94]  Farzad Khalvati,et al.  Radiomics analysis at PET/CT contributes to prognosis of recurrence and survival in lung cancer treated with stereotactic body radiotherapy , 2018, Scientific Reports.

[95]  Raymond H Mak,et al.  CT-based radiomic analysis of stereotactic body radiation therapy patients with lung cancer. , 2016, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[96]  Z. Tang,et al.  Value of texture analysis based on enhanced MRI for predicting an early therapeutic response to transcatheter arterial chemoembolisation combined with high-intensity focused ultrasound treatment in hepatocellular carcinoma. , 2018, Clinical radiology.

[97]  Fei Yang,et al.  Evaluation of radiomic texture feature error due to MRI acquisition and reconstruction: A simulation study utilizing ground truth. , 2018, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[98]  Bruce R. Rosen,et al.  Distributed deep learning networks among institutions for medical imaging , 2018, J. Am. Medical Informatics Assoc..

[99]  P. Lambin,et al.  Stability of FDG-PET Radiomics features: An integrated analysis of test-retest and inter-observer variability , 2013, Acta oncologica.

[100]  I. El Naqa,et al.  A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities , 2015, Physics in medicine and biology.

[101]  Benjamin Movsas,et al.  On the impact of smoothing and noise on robustness of CT and CBCT radiomics features for patients with head and neck cancers , 2017, Medical physics.

[102]  Milan Sonka,et al.  3D Slicer as an image computing platform for the Quantitative Imaging Network. , 2012, Magnetic resonance imaging.

[103]  M. Giger,et al.  Breast image analysis for risk assessment, detection, diagnosis, and treatment of cancer. , 2013, Annual review of biomedical engineering.

[104]  Hans Meine,et al.  Automatic liver tumor segmentation in CT with fully convolutional neural networks and object-based postprocessing , 2018, Scientific Reports.

[105]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[106]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[107]  Daniel B. Mark,et al.  TUTORIAL IN BIOSTATISTICS MULTIVARIABLE PROGNOSTIC MODELS: ISSUES IN DEVELOPING MODELS, EVALUATING ASSUMPTIONS AND ADEQUACY, AND MEASURING AND REDUCING ERRORS , 1996 .

[108]  Robert J. Gillies,et al.  Quantitative imaging of cancer in the postgenomic era: Radio(geno)mics, deep learning, and habitats , 2018, Cancer.

[109]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[110]  Masoom A. Haider,et al.  Automated prostate cancer detection via comprehensive multi-parametric magnetic resonance imaging texture feature models , 2015, BMC Medical Imaging.

[111]  Thomas Carlier,et al.  Revisiting the Robustness of PET-Based Textural Features in the Context of Multi-Centric Trials , 2016, PloS one.

[112]  Laurence E. Court,et al.  The utility of quantitative CT radiomics features for improved prediction of radiation pneumonitis , 2018, Medical physics.

[113]  P. Lambin,et al.  Robust Radiomics Feature Quantification Using Semiautomatic Volumetric Segmentation , 2014, PloS one.

[114]  Dimitris Visvikis,et al.  Characterization of PET/CT images using texture analysis: the past, the present… any future? , 2016, European Journal of Nuclear Medicine and Molecular Imaging.

[115]  Raymond H Mak,et al.  Radiomic phenotype features predict pathological response in non-small cell lung cancer. , 2016, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[116]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[117]  R. Jeraj,et al.  Variability of textural features in FDG PET images due to different acquisition modes and reconstruction parameters , 2010, Acta oncologica.

[118]  Peter Balter,et al.  Delta-radiomics features for the prediction of patient outcomes in non–small cell lung cancer , 2017, Scientific Reports.

[119]  Sanjay N. Talbar,et al.  Radiomics based detection and characterization of suspicious lesions on full field digital mammograms , 2018, Comput. Methods Programs Biomed..

[120]  Oscar Acosta,et al.  Haralick textural features on T2‐weighted MRI are associated with biochemical recurrence following radiotherapy for peripheral zone prostate cancer , 2017, Journal of magnetic resonance imaging : JMRI.

[121]  R. Gillies,et al.  Deep learning for lung cancer prognostication: A retrospective multi-cohort radiomics study , 2018, PLoS medicine.

[122]  Jinzhong Yang,et al.  Measuring Computed Tomography Scanner Variability of Radiomics Features , 2015, Investigative radiology.

[123]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[124]  Aaron D Ward,et al.  Detection of Local Cancer Recurrence After Stereotactic Ablative Radiation Therapy for Lung Cancer: Physician Performance Versus Radiomic Assessment. , 2016, International journal of radiation oncology, biology, physics.

[125]  Stephen S F Yip,et al.  Radiomics‐based Assessment of Radiation‐induced Lung Injury After Stereotactic Body Radiotherapy , 2017, Clinical lung cancer.

[126]  Anant Madabhushi,et al.  Identifying quantitative in vivo multi-parametric MRI features for treatment related changes after laser interstitial thermal therapy of prostate cancer , 2014, Neurocomputing.

[127]  Anant Madabhushi,et al.  Radiomic features on MRI enable risk categorization of prostate cancer patients on active surveillance: Preliminary findings , 2018, Journal of magnetic resonance imaging : JMRI.

[128]  Andriy Fedorov,et al.  Computational Radiomics System to Decode the Radiographic Phenotype. , 2017, Cancer research.

[129]  P. Lambin,et al.  Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach , 2014, Nature Communications.

[130]  Martin Sill,et al.  Large-scale Radiomic Profiling of Recurrent Glioblastoma Identifies an Imaging Predictor for Stratifying Anti-Angiogenic Treatment Response , 2016, Clinical Cancer Research.

[131]  I. Sohn,et al.  Imaging Phenotyping Using Radiomics to Predict Micropapillary Pattern within Lung Adenocarcinoma , 2017, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[132]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.