Automated classification of multiphoton microscopy images of ovarian tissue using deep learning

Abstract. Histopathological image analysis of stained tissue slides is routinely used in tumor detection and classification. However, diagnosis requires a highly trained pathologist and can thus be time-consuming, labor-intensive, and potentially risk bias. Here, we demonstrate a potential complementary approach for diagnosis. We show that multiphoton microscopy images from unstained, reproductive tissues can be robustly classified using deep learning techniques. We fine-train four pretrained convolutional neural networks using over 200 murine tissue images based on combined second-harmonic generation and two-photon excitation fluorescence contrast, to classify the tissues either as healthy or associated with high-grade serous carcinoma with over 95% sensitivity and 97% specificity. Our approach shows promise for applications involving automated disease diagnosis. It could also be readily applied to other tissues, diseases, and related classification problems.

[1]  Stephen T. C. Wong,et al.  Combining deep learning and coherent anti-Stokes Raman scattering imaging for automated differential diagnosis of lung cancer , 2017, Journal of biomedical optics.

[2]  Rebecca L. Siegel Mph,et al.  Cancer statistics, 2016 , 2016 .

[3]  A. Fabre,et al.  Imaging lipid bodies in cells and tissues using third-harmonic generation microscopy , 2005, Nature Methods.

[4]  W. Zipfel,et al.  Strategies for high-resolution imaging of epithelial ovarian cancer by laparoscopic nonlinear microscopy. , 2010, Translational oncology.

[5]  C. Rueden,et al.  Bmc Medicine Collagen Density Promotes Mammary Tumor Initiation and Progression , 2022 .

[6]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Jia Deng,et al.  A large-scale hierarchical image database , 2009, CVPR 2009.

[8]  W. Denk,et al.  Two-photon laser scanning fluorescence microscopy. , 1990, Science.

[9]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[10]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[11]  Richard A. Szucs,et al.  TNM Classification of Malignant Tumors. 5th ed , 1998 .

[12]  J. Prat New insights into ovarian cancer pathology. , 2012, Annals of oncology : official journal of the European Society for Medical Oncology.

[13]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[14]  Z. Werb,et al.  Remodelling the extracellular matrix in development and disease , 2014, Nature Reviews Molecular Cell Biology.

[15]  A. Jemal,et al.  Cancer statistics, 2016 , 2016, CA: a cancer journal for clinicians.

[16]  Nima Tajbakhsh,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE Transactions on Medical Imaging.

[17]  Vikas Singh,et al.  3D texture analysis for classification of second harmonic generation images of human ovarian cancer , 2016, Scientific Reports.

[18]  B. Vanderhyden,et al.  A New Spontaneously Transformed Syngeneic Model of High-Grade Serous Ovarian Cancer with a Tumor-Initiating Cell Population , 2014, Front. Oncol..

[19]  Urs Utzinger,et al.  Endogenous Optical Biomarkers of Ovarian Cancer Evaluated with Multiphoton Microscopy , 2007, Cancer Epidemiology Biomarkers & Prevention.

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Paul Campagnola,et al.  Second harmonic generation imaging microscopy: applications to diseases diagnostics. , 2011, Analytical chemistry.

[22]  Urs Utzinger,et al.  In vivo time-serial multi-modality optical imaging in a mouse model of ovarian tumorigenesis , 2014, Cancer biology & therapy.

[23]  Hayit Greenspan,et al.  Deep learning with non-medical training used for chest pathology identification , 2015, Medical Imaging.

[24]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[25]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[26]  Vikas Singh,et al.  Texture analysis applied to second harmonic generation image data for ovarian cancer classification , 2014, Journal of biomedical optics.

[27]  Gert-Jan Bakker,et al.  Third harmonic generation microscopy of cells and tissue organization , 2016, Journal of Cell Science.

[28]  A. Pena,et al.  Second harmonic imaging and scoring of collagen in fibrotic tissues. , 2007, Optics express.

[29]  J. Bellanger,et al.  Determination of extracellular matrix collagen fibril architectures and pathological remodeling by polarization dependent second harmonic microscopy , 2017, Scientific Reports.

[30]  Thomas R. Cox,et al.  Remodeling and homeostasis of the extracellular matrix: implications for fibrotic diseases and cancer , 2011, Disease Models & Mechanisms.

[31]  Mark Hewko,et al.  Collagen morphology and texture analysis: from statistics to classification , 2013, Scientific Reports.

[32]  Dayong Wang,et al.  Deep Learning for Identifying Metastatic Breast Cancer , 2016, ArXiv.

[33]  Patrick Haffner,et al.  Support vector machines for histogram-based image classification , 1999, IEEE Trans. Neural Networks.

[34]  Virginijus Barzda,et al.  Characterization of collagen in non-small cell lung carcinoma with second harmonic polarization microscopy. , 2014, Biomedical optics express.

[35]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[36]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[37]  Oleg Nadiarnykh,et al.  Alterations of the extracellular matrix in ovarian cancer studied by Second Harmonic Generation imaging microscopy , 2010, BMC Cancer.

[38]  M. Kauranen,et al.  Chiral imaging of collagen by second-harmonic generation circular dichroism , 2013, Biomedical optics express.