Combining deep learning and coherent anti-Stokes Raman scattering imaging for automated differential diagnosis of lung cancer

Abstract. Lung cancer is the most prevalent type of cancer and the leading cause of cancer-related deaths worldwide. Coherent anti-Stokes Raman scattering (CARS) is capable of providing cellular-level images and resolving pathologically related features on human lung tissues. However, conventional means of analyzing CARS images requires extensive image processing, feature engineering, and human intervention. This study demonstrates the feasibility of applying a deep learning algorithm to automatically differentiate normal and cancerous lung tissue images acquired by CARS. We leverage the features learned by pretrained deep neural networks and retrain the model using CARS images as the input. We achieve 89.2% accuracy in classifying normal, small-cell carcinoma, adenocarcinoma, and squamous cell carcinoma lung images. This computational method is a step toward on-the-spot diagnosis of lung cancer and can be further strengthened by the efforts aimed at miniaturizing the CARS technique for fiber-based microendoscopic imaging.

[1]  H. Abdi.  Partial Least Square Regression PLS-Regression , 2007 .

[2]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[3]  Stephen T. C. Wong,et al.  Multimodal non-linear optical imaging for label-free differentiation of lung cancerous lesions from normal and desmoplastic tissues. , 2013, Biomedical optics express.

[4]  Calum MacAulay,et al.  Innovative molecular and imaging approaches for the detection of lung cancer and its precursor lesions , 2002, Oncogene.

[5]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  D. Shen,et al.  Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans , 2016, Scientific Reports.

[7]  T. Allen,et al.  Color atlas and text of pulmonary pathology , 2008 .

[8]  Bhiksha Raj,et al.  On the Origin of Deep Learning , 2017, ArXiv.

[9]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[10]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[11]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[12]  Stephen T. C. Wong,et al.  Delivery of picosecond lasers in multimode fibers for coherent anti-Stokes Raman scattering imaging. , 2010, Optics express.

[13]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[14]  L. Freitag,et al.  Optimized autofluorescence bronchoscopy using additional backscattered red light. , 2007, Journal of biomedical optics.

[15]  Jane A Dickerson,et al.  Current Applications of Liquid Chromatography / Mass Spectrometry in Pharmaceutical Discovery After a Decade of Innovation , 2008 .

[16]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[17]  H Suzuki,et al.  The changes of the stromal elastotic framework in the growth of peripheral lung adenocarcinomas , 1996, Cancer.

[18]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[19]  Stephen T. C. Wong,et al.  Multimodal nonlinear endo-microscopy probe design for high resolution, label-free intraoperative imaging. , 2015, Biomedical optics express.

[20]  Andreas Volkmer,et al.  Theoretical and experimental characterization of coherent anti-Stokes Raman scattering microscopy , 2002 .

[21]  H. Lynch,et al.  Psychologic Aspects of Cancer Genetic Testing: A Research Update for Clinicians , 1997 .

[22]  Stephen T. C. Wong,et al.  Differential diagnosis of lung carcinoma with three-dimensional quantitative molecular vibrational imaging. , 2012, Journal of biomedical optics.

[23]  X. Xie,et al.  Coherent Anti-Stokes Raman Scattering Microscopy: Instrumentation, Theory, and Applications , 2004 .

[24]  Stephen T. C. Wong,et al.  Use of multimode optical fibers for fiber-based coherent anti-Stokes Raman scattering microendoscopy imaging. , 2011, Optics Letters.

[25]  M. Toshima,et al.  Three-dimensional architecture of elastin and collagen fiber networks in the human and rat lung. , 2004, Archives of histology and cytology.

[26]  Konstantinos Zarogoulidis,et al.  Advanced bronchoscopic techniques in diagnosis and staging of lung cancer. , 2013, Journal of thoracic disease.

[27]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[28]  Stephen T. C. Wong,et al.  Label-free high-resolution imaging of prostate glands and cavernous nerves using coherent anti-Stokes Raman scattering microscopy , 2011, Biomedical optics express.

[29]  Stephen T. C. Wong,et al.  Differential diagnosis of breast cancer using quantitative, label-free and molecular vibrational imaging , 2011, Biomedical optics express.

[30]  S. Kanemaki,et al.  A Theory for the , 1986 .

[31]  A. Jemal,et al.  Cancer statistics, 2017 , 2017, CA: a cancer journal for clinicians.

[32]  Paul Babyn,et al.  Thyroid Nodule Classification in Ultrasound Images by Fine-Tuning Deep Convolutional Neural Network , 2017, Journal of Digital Imaging.

[33]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[34]  Stephen T. C. Wong,et al.  Dual CARS and SHG image acquisition scheme that combines single central fiber and multimode fiber bundle to collect and differentiate backward and forward generated photons. , 2016, Biomedical optics express.

[35]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[36]  G. Ripandelli,et al.  Optical coherence tomography. , 1998, Seminars in ophthalmology.

[37]  Samuel Leung,et al.  Optimal Immunohistochemical Markers For Distinguishing Lung Adenocarcinomas From Squamous Cell Carcinomas in Small Tumor Samples , 2010, The American journal of surgical pathology.

[38]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[39]  Mbbs Md FRCPath Donald N. Pritzker Vinay Kumar.  Robbins and Cotran pathologic basis of disease , 2015 .

[40]  R. Khokha,et al.  Molecular mechanisms of tumor invasion and metastasis: an integrated view. , 2003, Current molecular medicine.

[41]  Stephen T. C. Wong,et al.  Differential diagnosis of lung carcinoma with coherent anti-Stokes Raman scattering imaging. , 2012, Archives of pathology & laboratory medicine.

[42]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[43]  S. Süsstrunk,et al.  SLIC Superpixels ? , 2010 .

[44]  Stephen T. C. Wong,et al.  On-the-spot lung cancer differential diagnosis by label-free, molecular vibrational imaging and knowledge-based classification. , 2011, Journal of biomedical optics.

[45]  O. Miettinen,et al.  Survival of Patients with Stage I Lung Cancer Detected on CT Screening , 2008 .

[46]  Hans C Gerritsen,et al.  Advances and challenges in label-free nonlinear optical imaging using two-photon excitation fluorescence and second harmonic generation for cancer research. , 2014, Journal of photochemistry and photobiology. B, Biology.

[47]  Pascal Fua,et al.  Supervoxel-Based Segmentation of Mitochondria in EM Image Stacks With Learned Shape Features , 2012, IEEE Transactions on Medical Imaging.

[48]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[49]  Edward L Spitznagel,et al.  Changing epidemiology of small-cell lung cancer in the United States over the last 30 years: analysis of the surveillance, epidemiologic, and end results database. , 2006, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[50]  Conor L Evans,et al.  Chemical imaging of tissue in vivo with video-rate coherent anti-Stokes Raman scattering microscopy. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[51]  Majid Naji,et al.  Miniaturized multimodal CARS microscope based on MEMS scanning and a single laser source. , 2010, Optics express.

[52]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[53]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[54]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Robert Hecht-Nielsen,et al.  Theory of the backpropagation neural network , 1989, International 1989 Joint Conference on Neural Networks.

[56]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[57]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[58]  Philip T Cagle,et al.  Revolution in lung cancer: new challenges for the surgical pathologist. , 2011, Archives of pathology & laboratory medicine.

[59]  Stephen T. C. Wong,et al.  Coherent anti-Stokes Raman scattering microscopy imaging with suppression of four-wave mixing in optical fibers. , 2011, Optics express.

[60]  O. Miettinen,et al.  Early Lung Cancer Action Project: overall design and findings from baseline screening , 1999, The Lancet.

[61]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[62]  X. Xie,et al.  Towards CARS Endoscopy. , 2006, Optics express.

[63]  Andreas Uhl,et al.  Transfer Learning for Colonic Polyp Classification Using Off-the-Shelf CNN Features , 2016, CARE@MICCAI.

[64]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[65]  Antoine Pariente,et al.  Factors influencing accuracy of CT-guided percutaneous biopsies of pulmonary lesions , 2004, European Radiology.

[66]  W. Denk,et al.  Two-photon laser scanning fluorescence microscopy. , 1990, Science.

[67]  Hod Lipson,et al.  Understanding Neural Networks Through Deep Visualization , 2015, ArXiv.

[68]  Zhiyong Wang,et al.  Automated Nuclear Segmentation of Coherent Anti-Stokes Raman Scattering Microscopy Images by Coupling Superpixel Context Information with Artificial Neural Networks , 2011, MLMI.

[69]  X. Xie,et al.  Coherent Raman scanning fiber endoscopy. , 2011, Optics letters.

[70]  Susanna M Cramb,et al.  The International Epidemiology of Lung Cancer: Geographical Distribution and Secular Trends , 2008, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[71]  Lida P. Hariri,et al.  Diagnosing lung carcinomas with optical coherence tomography. , 2015, Annals of the American Thoracic Society.

[72]  Tianbao Yang,et al.  Hyper-class augmented and regularized deep learning for fine-grained image classification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[73]  Calum MacAulay,et al.  Coregistered autofluorescence-optical coherence tomography imaging of human lung sections , 2014, Journal of biomedical optics.

[74]  Ming-Sound Tsao,et al.  Low Prevalence of High-Grade Lesions Detected With Autofluorescence Bronchoscopy in the Setting of Lung Cancer Screening in the Pan-Canadian Lung Cancer Screening Study. , 2015, Chest.

[75]  Michael I. Jordan,et al.  The Handbook of Brain Theory and Neural Networks , 2002 .

[76]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[77]  Jia Deng,et al.  A large-scale hierarchical image database , 2009, CVPR 2009.

[78]  Gangjun Liu,et al.  Fiber delivered probe for efficient CARS imaging of tissues. , 2010, Optics express.