Assessing robustness of radiomic features by image perturbation

Image features need to be robust against differences in positioning, acquisition and segmentation to ensure reproducibility. Radiomic models that only include robust features can be used to analyse new images, whereas models with non-robust features may fail to predict the outcome of interest accurately. Test-retest imaging is recommended to assess robustness, but may not be available for the phenotype of interest. We therefore investigated 18 combinations of image perturbations to determine feature robustness, based on noise addition (N), translation (T), rotation (R), volume growth/shrinkage (V) and supervoxel-based contour randomisation (C). Test-retest and perturbation robustness were compared for combined total of 4032 morphological, statistical and texture features that were computed from the gross tumour volume in two cohorts with computed tomography imaging: I) 31 non-small-cell lung cancer (NSCLC) patients; II): 19 head-and-neck squamous cell carcinoma (HNSCC) patients. Robustness was determined using the 95% confidence interval (CI) of the intraclass correlation coefficient (1, 1). Features with CI ≥ 0:90 were considered robust. The NTCV, TCV, RNCV and RCV perturbation chain produced similar results and identified the fewest false positive robust features (NSCLC: 0.2–0.9%; HNSCC: 1.7–1.9%). Thus, these perturbation chains may be used as an alternative to test-retest imaging to assess feature robustness.

[1]  Binsheng Zhao,et al.  Evaluating variability in tumor measurements from same-day repeat CT scans of patients with non-small cell lung cancer. , 2009, Radiology.

[2]  G. Collewet,et al.  Influence of MRI acquisition protocols and image intensity normalization methods on texture classification. , 2004, Magnetic resonance imaging.

[3]  Laurence Court,et al.  Effect of tube current on computed tomography radiomic features , 2018, Scientific Reports.

[4]  K. Jarrod Millman,et al.  Python for Scientists and Engineers , 2011, Comput. Sci. Eng..

[5]  R. Steenbakkers,et al.  The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping. , 2020, Radiology.

[6]  I. El Naqa,et al.  A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities , 2015, Physics in medicine and biology.

[7]  Terry K Koo,et al.  A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. , 2016, Journal Chiropractic Medicine.

[8]  Mitsuru Ikeda,et al.  A method for estimating noise variance of CT image , 2010, Comput. Medical Imaging Graph..

[9]  P. Lambin,et al.  Radiomics: the bridge between medical imaging and personalized medicine , 2017, Nature Reviews Clinical Oncology.

[10]  J. Steinbach,et al.  Residual tumour hypoxia in head-and-neck cancer patients undergoing primary radiochemotherapy, final results of a prospective trial on repeat FMISO-PET imaging. , 2017, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[11]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[12]  Travis E. Oliphant,et al.  Python for Scientific Computing , 2007, Computing in Science & Engineering.

[13]  Zaiyi Liu,et al.  Effects of contrast-enhancement, reconstruction slice thickness and convolution kernel on the diagnostic performance of radiomics signature in solitary pulmonary nodule , 2016, Scientific Reports.

[14]  M. Hatt,et al.  Responsible Radiomics Research for Faster Clinical Translation , 2017, The Journal of Nuclear Medicine.

[15]  O. Riesterer,et al.  Stability of radiomic features in CT perfusion maps , 2016, Physics in medicine and biology.

[16]  Martin Vetterli,et al.  Adaptive wavelet thresholding for image denoising and compression , 2000, IEEE Trans. Image Process..

[17]  J. E. van Timmeren,et al.  Influence of gray level discretization on radiomic feature stability for different CT scanners, tube currents and slice thicknesses: a comprehensive phantom study , 2017, Acta oncologica.

[18]  H. Aerts,et al.  Applications and limitations of radiomics , 2016, Physics in medicine and biology.

[19]  R. Gillies,et al.  Repeatability and Reproducibility of Radiomic Features: A Systematic Review , 2018, International journal of radiation oncology, biology, physics.

[20]  El Naqa,et al.  A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities , 2015 .

[21]  Derek C. Rose,et al.  Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier] , 2010, IEEE Computational Intelligence Magazine.

[22]  Jinzhong Yang,et al.  Measuring Computed Tomography Scanner Variability of Radiomics Features , 2015, Investigative radiology.

[23]  Thomas Lewiner,et al.  Efficient Implementation of Marching Cubes' Cases with Topological Guarantees , 2003, J. Graphics, GPU, & Game Tools.

[24]  M. Hatt,et al.  18F-FDG PET Uptake Characterization Through Texture Analysis: Investigating the Complementary Nature of Heterogeneity and Functional Tumor Volume in a Multi–Cancer Site Patient Cohort , 2015, The Journal of Nuclear Medicine.

[25]  Samuel H. Hawkins,et al.  Reproducibility and Prognosis of Quantitative Features Extracted from CT Images. , 2014, Translational oncology.

[26]  Ronald Boellaard,et al.  Repeatability of Radiomic Features in Non-Small-Cell Lung Cancer [18F]FDG-PET/CT Studies: Impact of Reconstruction and Delineation , 2016, Molecular Imaging and Biology.

[27]  Jiazhou Wang,et al.  Test–Retest Data for Radiomics Feature Stability Analysis: Generalizable or Study-Specific? , 2016, Tomography.

[28]  P. Lambin,et al.  Stability of FDG-PET Radiomics features: An integrated analysis of test-retest and inter-observer variability , 2013, Acta oncologica.

[29]  Issam El-Naqa,et al.  Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer , 2017, Scientific Reports.

[30]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[31]  Steffen Löck,et al.  Image biomarker standardisation initiative , 2016 .

[32]  FuaPascal,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012 .

[33]  Laurence Court,et al.  Harmonizing the pixel size in retrospective computed tomography radiomics studies , 2017, PloS one.

[34]  W. Tsai,et al.  Reproducibility of radiomics for deciphering tumor phenotype with imaging , 2016, Scientific Reports.

[35]  Patrick Granton,et al.  Radiomics: extracting more information from medical images using advanced feature analysis. , 2012, European journal of cancer.

[36]  Thomas Frauenfelder,et al.  Influence of inter-observer delineation variability on radiomics stability in different tumor sites , 2018, Acta oncologica.

[37]  Mithat Gönen,et al.  Influence of CT acquisition and reconstruction parameters on radiomic feature reproducibility , 2018, Journal of medical imaging.

[38]  Andre Dekker,et al.  Radiomics: the process and the challenges. , 2012, Magnetic resonance imaging.

[39]  Stephen M. Moore,et al.  The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository , 2013, Journal of Digital Imaging.

[40]  William E. Lorensen,et al.  Marching cubes: a high resolution 3D surface construction algorithm , 1996 .

[41]  Steffen Löck,et al.  A comparative study of machine learning methods for time-to-event survival data for radiomics risk modelling , 2017, Scientific Reports.

[42]  M. Hatt,et al.  Reproducibility of Tumor Uptake Heterogeneity Characterization Through Textural Feature Analysis in 18F-FDG PET , 2012, The Journal of Nuclear Medicine.

[43]  Emmanuelle Gouillart,et al.  scikit-image: image processing in Python , 2014, PeerJ.

[44]  Wolfgang Weber,et al.  Reliability of PET/CT Shape and Heterogeneity Features in Functional and Morphologic Components of Non–Small Cell Lung Cancer Tumors: A Repeatability Analysis in a Prospective Multicenter Cohort , 2016, The Journal of Nuclear Medicine.

[45]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Geoffrey G. Zhang,et al.  Intrinsic dependencies of CT radiomic features on voxel size and number of gray levels , 2017, Medical physics.