Reproducible Machine Learning Methods for Lung Cancer Detection Using Computed Tomography Images: Algorithm Development and Validation

Background Chest computed tomography (CT) is crucial for the detection of lung cancer, and many automated CT evaluation methods have been proposed. Due to the divergent software dependencies of the reported approaches, the developed methods are rarely compared or reproduced. Objective The goal of the research was to generate reproducible machine learning modules for lung cancer detection and compare the approaches and performances of the award-winning algorithms developed in the Kaggle Data Science Bowl. Methods We obtained the source codes of all award-winning solutions of the Kaggle Data Science Bowl Challenge, where participants developed automated CT evaluation methods to detect lung cancer (training set n=1397, public test set n=198, final test set n=506). The performance of the algorithms was evaluated by the log-loss function, and the Spearman correlation coefficient of the performance in the public and final test sets was computed. Results Most solutions implemented distinct image preprocessing, segmentation, and classification modules. Variants of U-Net, VGGNet, and residual net were commonly used in nodule segmentation, and transfer learning was used in most of the classification algorithms. Substantial performance variations in the public and final test sets were observed (Spearman correlation coefficient = .39 among the top 10 teams). To ensure the reproducibility of results, we generated a Docker container for each of the top solutions. Conclusions We compared the award-winning algorithms for lung cancer detection and generated reproducible Docker images for the top solutions. Although convolutional neural networks achieved decent accuracy, there is plenty of room for improvement regarding model generalizability.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  R. Altman,et al.  Association of Omics Features with Histopathology Patterns in Lung Adenocarcinoma. , 2017, Cell systems.

[3]  A. Jemal,et al.  Global cancer statistics, 2012 , 2015, CA: a cancer journal for clinicians.

[4]  Geoffrey I. Webb,et al.  On the effect of data set size on bias and variance in classification learning , 1999 .

[5]  V. Moyer.  Screening for Lung Cancer: U.S. Preventive Services Task Force Recommendation Statement , 2014, Annals of Internal Medicine.

[6]  Anthony J. Sherbondy,et al.  Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. , 2005, Radiology.

[7]  Isaac S Kohane,et al.  Artificial Intelligence in Healthcare , 2019, Artificial Intelligence and Machine Learning for Business for Non-Engineers.

[8]  Christopher Ré,et al.  Classifying non-small cell lung cancer types and transcriptomic subtypes using convolutional neural networks , 2020, J. Am. Medical Informatics Assoc..

[9]  Tomohiro Kuroda,et al.  Computer-aided diagnosis of lung nodule classification between benign nodule, primary lung cancer, and metastatic lung cancer at different image size using deep convolutional neural network with transfer learning , 2018, PloS one.

[10]  Brahim Ait Skourt,et al.  Lung CT Image Segmentation Using Deep Neural Networks , 2018 .

[11]  Richard C. Pais,et al.  The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. , 2011, Medical physics.

[12]  C. Gatsonis,et al.  Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening , 2012 .

[13]  R. Young,et al.  Community Low-Dose CT Lung Cancer Screening: A Prospective Cohort Study , 2015, Lung.

[14]  D. Lynch,et al.  The National Lung Screening Trial: overview and study design. , 2011, Radiology.

[15]  David Lynch,et al.  Reader Variability in Identifying Pulmonary Nodules on Chest Radiographs From the National Lung Screening Trial , 2012, Journal of thoracic imaging.

[16]  D. Naidich,et al.  Evaluation of individuals with pulmonary nodules: when is it lung cancer? Diagnosis and management of lung cancer, 3rd ed: American College of Chest Physicians evidence-based clinical practice guidelines. , 2013, Chest.

[17]  John W. McBride,et al.  The Application of Voxel Size Correction in X-ray Computed Tomography for Dimensional Metrology , 2013 .

[18]  K. Awai,et al.  Pulmonary nodules at chest CT: effect of computer-aided diagnosis on radiologists' detection performance. , 2004, Radiology.

[19]  N. Arunkumar,et al.  Optimal deep learning model for classification of lung cancer on CT images , 2019, Future Gener. Comput. Syst..

[20]  Yongdong Zhang,et al.  Automated pulmonary nodule detection in CT images using deep convolutional neural networks , 2019, Pattern Recognit..

[21]  Michael Snyder,et al.  Omics AnalySIs System for PRecision Oncology (OASISPRO): a web-based omics analysis tool for clinical phenotype prediction , 2018, Bioinform..

[22]  Kunio Doi,et al.  Computer-aided diagnosis in medical imaging: Historical review, current status and future potential , 2007, Comput. Medical Imaging Graph..

[23]  I. Kohane,et al.  Framing the challenges of artificial intelligence in medicine , 2018, BMJ Quality & Safety.

[24]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[25]  Carl Boettiger,et al.  An introduction to Docker for reproducible research , 2014, OPSR.

[26]  A. Jemal,et al.  Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries , 2018, CA: a cancer journal for clinicians.

[27]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Anselmo Cardoso de Paiva,et al.  Convolutional neural network-based PSO for lung nodule false positive reduction on CT images , 2018, Comput. Methods Programs Biomed..

[29]  Berkman Sahiner,et al.  Lung nodule detection on thoracic computed tomography images: preliminary evaluation of a computer-aided diagnosis system. , 2002, Medical physics.

[30]  Ce Zhang,et al.  Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features , 2016, Nature Communications.

[31]  Karen Drukker,et al.  LUNGx Challenge for computerized lung nodule classification , 2016, Journal of medical imaging.

[32]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[33]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[34]  Hao Chen,et al.  Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: The LUNA16 challenge , 2016, Medical Image Anal..