A deep separable neural network for human tissue identification in three-dimensional optical coherence tomography images

Abstract This research proposes a dilated depthwise separable network for human tissue identification using three-dimensional (3D) optical coherence tomography (OCT) images. Automatic human tissue identification has made it possible for fast pathological tissue analyses, detecting tissue changes over time, and efficiently making a precise therapy treatment plan. 3D medical image classification is a challenging task because of the indistinct tissue characteristics and computational efficiency. To address the challenge, a deep dilated depthwise separable convolutional neural network is proposed in this research. A depthwise separable architecture is introduced to improve parameter utilization efficiency. Dilated convolutions are applied to systematically aggregate multiscale contextual information and provide a large receptive field with a small number of trainable weights, which provide computational benefit. 2D convolutions are applied in the proposed model to enhance the computational efficiency. The constructed model is tested by performing a multi-class human thyroid tissue classification on 3D OCT images. For comparison, experimental results are obtained for texture feature-based shallow learning models and typical deep learning classification models. The results show that the proposed DDSCN model outperforms those state-of-art models and can improve accuracy by 3.2% compared with the best texture-based model and 2.27% compared with the best CNN model. The proposed deep model provides significant potential for the applicability of the deep learning technique to analyze medical images of human tissue while advancing the next generation of OCT-based real-time surgery image guidance.

[1]  Jerry L Prince,et al.  Retinal layer segmentation of macular OCT images using boundary classification , 2013, Biomedical optics express.

[2]  Sang Won Yoon,et al.  A Dual-Tree Complex Wavelet Transform Based Convolutional Neural Network for Human Thyroid Medical Image Segmentation , 2018, 2018 IEEE International Conference on Healthcare Informatics (ICHI).

[3]  Katherine M. Li,et al.  Skin Lesion Analysis Towards Melanoma Detection via End-to-end Deep Learning of Convolutional Neural Networks , 2018, ArXiv.

[4]  Bianca S. Gerendas,et al.  Supervised learning and dimension reduction techniques for quantification of retinal fluid in optical coherence tomography images , 2017, Eye.

[5]  Andrew M. Rollins,et al.  Integrative Advances for OCT-Guided Ophthalmic Surgery and Intraoperative OCT: Microscope Integration, Surgical Instrumentation, and Heads-Up Display Surgeon Feedback , 2014, 2015 Conference on Lasers and Electro-Optics (CLEO).

[6]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[7]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Michael J. A. Girard,et al.  A Deep Learning Approach to Digitally Stain Optical Coherence Tomography Images of the Optic Nerve Head , 2017, Investigative ophthalmology & visual science.

[9]  Chong Wang,et al.  Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search. , 2017, Biomedical optics express.

[10]  Sumeet Dua,et al.  Segmentation of Fluorescence Microscopy Cell Images Using Unsupervised Mining , 2010, The open medical informatics journal.

[11]  Christian Dannecker,et al.  Optical coherence tomography for the diagnosis of cervical intraepithelial neoplasia , 2011, Lasers in surgery and medicine.

[12]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[13]  F. Zhou,et al.  Machine learning based detection of age-related macular degeneration (AMD) and diabetic macular edema (DME) from optical coherence tomography (OCT) images. , 2016, Biomedical optics express.

[14]  Alex Cable,et al.  Optical coherence tomography of human kidney. , 2010, The Journal of urology.

[15]  Hongki Yoo,et al.  Automated detection of vessel lumen and stent struts in intravascular optical coherence tomography to evaluate stent apposition and neointimal coverage. , 2016, Medical physics.

[16]  Bernhard Baumann,et al.  Optical Coherence Tomography for Brain Imaging , 2019 .

[17]  Yue Zhu,et al.  Rapid and high-resolution imaging of human liver specimens by full-field optical coherence tomography , 2015, Journal of biomedical optics.

[18]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Ivan W. Selesnick,et al.  Three Dimensional Data-Driven Multi Scale Atomic Representation of Optical Coherence Tomography , 2015, IEEE Transactions on Medical Imaging.

[20]  Alex Vitkin,et al.  Differential diagnosis of human bladder mucosa pathologies in vivo with cross-polarization optical coherence tomography. , 2015, Biomedical optics express.

[21]  Ryo Asaoka,et al.  Discriminating between Glaucoma and Normal Eyes Using Optical Coherence Tomography and the ‘Random Forests’ Classifier , 2014, PloS one.

[22]  C. Berking,et al.  High‐definition optical coherence tomography of melanocytic skin lesions , 2015, Journal of biophotonics.

[23]  Jing Jing,et al.  Machine learning-based detection and segmentation of bioresorbable vascular scaffolds struts in intravascular OCT images , 2017 .

[24]  Robert A. McLaughlin,et al.  A review of optical coherence tomography in breast cancer , 2014 .

[25]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Yongyang Huang,et al.  Optical Coherence Tomography for Brain Imaging and Developmental Biology , 2016, IEEE Journal of Selected Topics in Quantum Electronics.

[27]  Hao Wu,et al.  Voxel-based plaque classification in coronary intravascular optical coherence tomography images using decision trees , 2018, Medical Imaging.

[28]  A. Senchenkov,et al.  Ultrasound in head and neck surgery: thyroid, parathyroid, and cervical lymph nodes. , 2004, The Surgical clinics of North America.

[29]  A. M. Leone,et al.  Plaque rupture and intact fibrous cap assessed by optical coherence tomography portend different outcomes in patients with acute coronary syndrome. , 2015, European heart journal.

[30]  Josien P. W. Pluim,et al.  Isointense infant brain MRI segmentation with a dilated convolutional neural network , 2017, ArXiv.

[31]  Christian Igel,et al.  Deep Feature Learning for Knee Cartilage Segmentation Using a Triplanar Convolutional Neural Network , 2013, MICCAI.

[32]  Jennifer K. Barton,et al.  Evaluation of segmentation algorithms for optical coherence tomography images of ovarian tissue , 2018, BiOS.

[33]  Frans Coenen,et al.  Dictionary Learning-Based Volumetric Image Classification for the Diagnosis of Age-Related Macular Degeneration , 2014, MLDM.

[34]  Sarah J. Erickson-Bhatt,et al.  Intraoperative optical coherence tomography of the human thyroid: Feasibility for surgical assessment , 2017, Translational research : the journal of laboratory and clinical medicine.

[35]  Kenneth Revett,et al.  Computer-aided diagnosis of human brain tumor through MRI: A survey and a new algorithm , 2014, Expert Syst. Appl..

[36]  Hsiang-Chieh Lee,et al.  Integrated optical coherence tomography and optical coherence microscopy imaging of ex vivo human renal tissues. , 2012, The Journal of urology.

[37]  Sina Farsiu,et al.  Fully automated detection of diabetic macular edema and dry age-related macular degeneration from optical coherence tomography images. , 2014, Biomedical optics express.

[38]  H. Stepp,et al.  Intraoperative optical coherence tomography imaging to identify parathyroid glands , 2015, Surgical Endoscopy.

[39]  Erping Long,et al.  Use of a Neural Net to Model the Impact of Optical Coherence Tomography Abnormalities on Vision in Age-related Macular Degeneration. , 2017, American journal of ophthalmology.

[40]  Heinz Handels,et al.  Segmentation of subcutaneous fat within mouse skin in 3D OCT image data using random forests , 2018, Medical Imaging.

[41]  Niharika Sachdeva,et al.  Stop the KillFies! Using Deep Learning Models to Identify Dangerous Selfies , 2018, WWW.

[42]  Xinjian Chen,et al.  Surrogate-Assisted Retinal OCT Image Classification Based on Convolutional Neural Networks , 2019, IEEE Journal of Biomedical and Health Informatics.

[43]  Sang Won Yoon,et al.  Medical Image Synthesis with Generative Adversarial Networks for Tissue Recognition , 2018, 2018 IEEE International Conference on Healthcare Informatics (ICHI).

[44]  Debjani Chakraborty,et al.  Transfer learning based classification of optical coherence tomography images with diabetic macular edema and dry age-related macular degeneration. , 2017, Biomedical optics express.

[45]  Mirza Faisal Beg,et al.  Retinal Fluid Segmentation and Detection in Optical Coherence Tomography Images using Fully Convolutional Neural Network , 2017, ArXiv.

[46]  B E Bouma,et al.  Optical biopsy in human gastrointestinal tissue using optical coherence tomography. , 1997, The American journal of gastroenterology.

[47]  Dalip Singh Mehta,et al.  In vivo classification of human skin burns using machine learning and quantitative features captured by optical coherence tomography , 2018 .

[48]  Jui-Kai Wang,et al.  Multimodal Segmentation of Optic Disc and Cup From SD-OCT and Color Fundus Photographs Using a Machine-Learning Graph-Based Approach , 2015, IEEE Transactions on Medical Imaging.

[49]  Alex Vitkin,et al.  Pixel classification method in optical coherence tomography for tumor segmentation and its complementary usage with OCT microangiography , 2018, Journal of biophotonics.

[50]  Nantheera Anantrasirichai,et al.  SVM-based texture classification in Optical Coherence Tomography , 2013, 2013 IEEE 10th International Symposium on Biomedical Imaging.

[51]  Peng Li,et al.  SVM Based Automatic Classification of Human Stomach Cancer with Optical Coherence Tomography Images , 2018, CLEO 2018.

[52]  William J Dupps,et al.  Determination of feasibility and utility of microscope-integrated optical coherence tomography during ophthalmic surgery: the DISCOVER Study RESCAN Results. , 2015, JAMA ophthalmology.

[53]  Milan Sonka,et al.  Geodesic Graph Cut Based Retinal Fluid Segmentation in Optical Coherence Tomography , 2015 .

[54]  Waseem Jerjes,et al.  Optical coherence tomography in the assessment of oral squamous cell carcinoma resection margins. , 2016, Photodiagnosis and photodynamic therapy.

[55]  Ruikang K. Wang,et al.  Theory, developments and applications of optical coherence tomography , 2005 .

[56]  Martin R. Hofmann,et al.  Ex vivo brain tumor analysis using spectroscopic optical coherence tomography , 2016, SPIE BiOS.

[57]  Yue Wu,et al.  Deep-Learning Based, Automated Segmentation of Macular Edema in Optical Coherence Tomography , 2017, bioRxiv.

[58]  Lida P. Hariri,et al.  Endobronchial Optical Coherence Tomography for Low‐Risk Microscopic Assessment and Diagnosis of Idiopathic Pulmonary Fibrosis In Vivo , 2017, American journal of respiratory and critical care medicine.

[59]  Nils Gessert,et al.  Force estimation from OCT volumes using 3D CNNs , 2018, International Journal of Computer Assisted Radiology and Surgery.

[60]  K. A. Vermeer,et al.  Automated segmentation by pixel classification of retinal layers in ophthalmic OCT images , 2011, Biomedical optics express.

[61]  Tsung-Han Tsai,et al.  Ex vivo imaging of human thyroid pathology using integrated optical coherence tomography and optical coherence microscopy. , 2010, Journal of biomedical optics.

[62]  J. Fujimoto,et al.  In vivo endoscopic optical biopsy with optical coherence tomography. , 1997, Science.

[63]  Michael D Abràmoff,et al.  A combined machine-learning and graph-based framework for the segmentation of retinal surfaces in SD-OCT volumes. , 2013, Biomedical optics express.

[64]  R. Mirnezami,et al.  Surgery 3.0, artificial intelligence and the next‐generation surgeon , 2018, The British journal of surgery.

[65]  Lindsey S. Folio,et al.  Three-Dimensional Spectral-Domain Optical Coherence Tomography Data Analysis for Glaucoma Detection , 2013, PloS one.

[66]  Yang Gao,et al.  Content-Based Image Retrieval Using Spatial Layout Information in Brain Tumor T1-Weighted Contrast-Enhanced MR Images , 2014, PloS one.

[67]  Williams,et al.  #165e Optical coherence tomography for the diagnosis of skin cancer in adults , 2018 .

[68]  Huikai Xie,et al.  Gastric and colon cancer imaging with swept source optical coherence tomography , 2017, 2017 Conference on Lasers and Electro-Optics Pacific Rim (CLEO-PR).

[69]  Neil J. Joshi,et al.  Automated diagnosis of myositis from muscle ultrasound: Exploring the use of machine learning and deep learning methods , 2017, PloS one.

[70]  Tien Yin Wong,et al.  Classification of SD-OCT volumes with multi pyramids, LBP and HOG descriptors: Application to DME detections , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[71]  J. Fujimoto,et al.  Optical coherence tomography as a method for identifying benign and malignant microscopic structures in the prostate gland. , 2000, Urology.

[72]  Simon K. Warfield,et al.  Asymmetric Loss Functions and Deep Densely-Connected Networks for Highly-Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection , 2018, IEEE Access.

[73]  Yu Gan,et al.  Automated classification of optical coherence tomography images of human atrial tissue , 2016, Journal of biomedical optics.

[74]  S. Boppart,et al.  Optical Coherence Tomography: Feasibility for Basic Research and Image-guided Surgery of Breast Cancer , 2004, Breast Cancer Research and Treatment.

[75]  D. Naidich,et al.  Computer-aided diagnosis and the evaluation of lung disease. , 2004, Journal of thoracic imaging.

[76]  L. Jampol,et al.  Optical coherence tomography use in evaluation of the vitreoretinal interface: a review. , 2007, Survey of ophthalmology.

[77]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[78]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[79]  Mirza Faisal Beg,et al.  Automatic detection of subretinal fluid and sub-retinal pigment epithelium fluid in optical coherence tomography images , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[80]  D. D. de Bruin,et al.  Optical coherence tomography for identification and quantification of human airway wall layers , 2017, PloS one.

[81]  Milan Sonka,et al.  Stratified Sampling Voxel Classification for Segmentation of Intraretinal and Subretinal Fluid in Longitudinal Clinical OCT Data , 2015, IEEE Transactions on Medical Imaging.

[82]  John S. Zelek,et al.  Classification of optical coherence tomography images for diagnosing different ocular diseases , 2018, BiOS.

[83]  Shweta Kharya,et al.  Using data mining techniques for diagnosis and prognosis of cancer disease , 2012, ArXiv.

[84]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[85]  H. Stepp,et al.  Optical coherence tomography as a method to identify parathyroid glands , 2013, Lasers in surgery and medicine.

[86]  Catarina Eloy,et al.  Classification of breast cancer histology images using Convolutional Neural Networks , 2017, PloS one.

[87]  Nils Gessert,et al.  A deep learning approach for pose estimation from volumetric OCT data , 2018, Medical Image Anal..

[88]  Tianheng Wang,et al.  An overview of optical coherence tomography for ovarian tissue imaging and characterization. , 2015, Wiley interdisciplinary reviews. Nanomedicine and nanobiotechnology.

[89]  Peter A. Calabresi,et al.  Topology guaranteed segmentation of the human retina from OCT using convolutional neural networks , 2018, ArXiv.

[90]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[91]  Antonio J. Serrano,et al.  A finite element-based machine learning approach for modeling the mechanical behavior of the breast tissues under compression in real-time , 2017, Comput. Biol. Medicine.

[92]  Sang Won Yoon,et al.  Breast cancer diagnosis based on feature extraction using a hybrid of K-means and support vector machine algorithms , 2014, Expert Syst. Appl..

[93]  Farhat Fnaiech,et al.  Color edges extraction using statistical features and automatic threshold technique: application to the breast cancer cells , 2014, Biomedical engineering online.

[94]  Hiroshi Mashimo,et al.  Assessment of the radiofrequency ablation dynamics of esophageal tissue with optical coherence tomography , 2017, Journal of biomedical optics.

[95]  E. McVeigh,et al.  Detection of human brain cancer infiltration ex vivo and in vivo using quantitative optical coherence tomography , 2015, Science Translational Medicine.

[96]  Boudewijn P F Lelieveldt,et al.  Noninvasive Detection of Metastases and Follicle Density in Ovarian Tissue Using Full-Field Optical Coherence Tomography , 2016, Clinical Cancer Research.

[97]  Marina Marjanovic,et al.  Differentiation of ex vivo human breast tissue using polarization-sensitive optical coherence tomography. , 2014, Biomedical optics express.

[98]  Kenneth K Wang,et al.  Optical Coherence Tomography: Clinical Applications in Gastrointestinal Endoscopy , 2016 .

[99]  T. Jørgensen,et al.  Optical coherence tomography in dermatology , 2006 .

[100]  Qin Huang,et al.  Structural markers observed with endoscopic 3-dimensional optical coherence tomography correlating with Barrett's esophagus radiofrequency ablation treatment response (with videos). , 2012, Gastrointestinal endoscopy.

[101]  Jinjun Xiong,et al.  Revisiting Pre-training: An Efficient Training Method for Image Classification. , 2018 .

[102]  Sang Won Yoon,et al.  A support vector machine-based ensemble algorithm for breast cancer diagnosis , 2017, Eur. J. Oper. Res..

[103]  Shusheng Bi,et al.  Segmentation of the Clustered Cells with Optimized Boundary Detection in Negative Phase Contrast Images , 2015, PloS one.

[104]  W. Desmet,et al.  Optical Coherence Tomography Findings in Patients With Coronary Stent Thrombosis , 2017, Circulation.