Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks

Intraoperative diagnosis is essential for providing safe and effective care during cancer surgery. The existing workflow for intraoperative diagnosis based on H&E staining of processed tissue is time-, resource-, and labor-intensive. Moreover, interpretation of intraoperative histologic images is dependent on a pathology workforce that is contracting and unevenly distributed across the centers where cancer surgery is performed worldwide. We developed an automated workflow, independent of traditional H&E histology, that combines stimulated Raman histology (SRH), a rapid label-free optical imaging method, and deep convolutional neural networks (CNN) to predict diagnosis at the bedside in near real time. Specifically, our CNN, trained on over 2.5 million SRH images, predicts brain tumor diagnosis in the operating room in under 150 s, which is an order of magnitude faster than conventional techniques (eg, 20-30 min). To validate our workflow in the clinical setting, we designed a multicenter, prospective, noninferiority clinical trial (N = 204) that compares SRH plus CNN vs traditional H&E histology. Primary endpoint was overall diagnostic accuracy. We show that CNN-based diagnosis of SRH images was noninferior to pathologist-based interpretation of conventional histologic images (overall accuracy, 94.6% vs 95.5%). Additionally, our CNN learned a hierarchy of interpretable histologic feature representations to classify the major histopathologic classes of brain tumors. We then developed and implemented a semantic segmentation method that can identify tumor infiltrated and diagnostic regions within SRH images. Mean intersection over union values was 61 ± 28.6 for ground truth diagnostic class and 86.0 ± 28.6 for tumor-infiltrated regions. We have demonstrated how combining bedside optical histology with deep learning can result in near real-time intraoperative brain tumor diagnosis. Our workflow provides a means of delivering expert-level intraoperative diagnosis where neuropathology resources are scarce and improve diagnostic accuracy in resource-rich centers.

[1]  Cary P Gross,et al.  Mohs micrographic surgery and surgical excision for nonmelanoma skin cancer treatment in the Medicare population. , 2012, Archives of dermatology.

[2]  Joel H. Saltz,et al.  Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  B. van Ginneken,et al.  Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis , 2016, Scientific Reports.

[4]  Mortazavi,et al.  Supporting Online Material Materials and Methods Figs. S1 to S13 Tables S1 to S3 References Label-free Biomedical Imaging with High Sensitivity by Stimulated Raman Scattering Microscopy , 2022 .

[5]  M. Snuderl,et al.  Rapid Intraoperative Diagnosis of Pediatric Brain Tumors Using Stimulated Raman Histology. , 2018, Cancer research.

[6]  Jill S Barnholtz-Sloan,et al.  CBTRUS Statistical Report: Primary brain and other central nervous system tumors diagnosed in the United States in 2010–2014 , 2017, Neuro-oncology.

[7]  G. Reifenberger,et al.  The 2016 World Health Organization Classification of Tumors of the Central Nervous System: a summary , 2016, Acta Neuropathologica.

[8]  V. P. Collins,et al.  Intratumor heterogeneity in human glioblastoma reflects cancer evolutionary dynamics , 2013, Proceedings of the National Academy of Sciences.

[9]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[10]  Umberto Veronesi,et al.  Global cancer surgery: delivering safe, affordable, and timely cancer surgery. , 2015, The Lancet. Oncology.

[11]  C. Carter,et al.  Relation of tumor size, lymph node status, and survival in 24,740 breast cancer cases , 1989, Cancer.

[12]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[13]  Johan M. Kros,et al.  Towards improving the safety and diagnostic yield of stereotactic biopsy in a single centre , 2010, Acta Neurochirurgica.

[14]  Todd C. Hollon,et al.  Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy , 2017, Nature Biomedical Engineering.

[15]  Isaiah Norton,et al.  Label-Free Neurosurgical Pathology with Stimulated Raman Imaging. , 2016, Cancer research.

[16]  A. Peet,et al.  Immunohistochemical staining of lipid droplets with adipophilin in paraffin-embedded glioma tissue identifies an association between lipid droplets and tumour grade , 2017 .

[17]  Philip T Cagle,et al.  The 100-year anniversary of the description of the frozen section procedure. , 2005, JAMA.

[18]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[19]  Susan Mallett,et al.  Intraoperative frozen section analysis for the diagnosis of early stage ovarian cancer in suspicious pelvic masses. , 2016, The Cochrane database of systematic reviews.

[20]  R. Zarbo,et al.  Interinstitutional comparison of frozen section turnaround time. A College of American Pathologists Q-Probes study of 32868 frozen sections in 700 hospitals. , 1997, Archives of pathology & laboratory medicine.

[21]  N. Razavian,et al.  Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning , 2018, Nature Medicine.

[22]  X. Xie,et al.  Stimulated Raman scattering microscopy with a robust fibre laser source , 2014, Nature Photonics.

[23]  Kibok Lee,et al.  A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks , 2018, NeurIPS.

[24]  Marcus A. Badgeley,et al.  Automated deep-neural-network surveillance of cranial images for acute neurologic events , 2018, Nature Medicine.

[25]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[26]  Timo Kohlberger,et al.  An augmented reality microscope with real-time artificial intelligence integration for cancer diagnosis , 2019, Nature Medicine.

[27]  J. M. Crawford,et al.  Pathologist workforce in the United States: I. Development of a predictive model to examine factors influencing supply. , 2013, Archives of pathology & laboratory medicine.

[28]  Aleksey Boyko,et al.  Detecting Cancer Metastases on Gigapixel Pathology Images , 2017, ArXiv.

[29]  J. Kuo,et al.  Detection of Human Brain Tumor Infiltration With Quantitative Stimulated Raman Scattering Microscopy. , 2016, Neurosurgery.

[30]  Chenchen Liu,et al.  How convolutional neural networks see the world - A survey of convolutional neural network visualization methods , 2018, Math. Found. Comput..