Deep learning based topology guaranteed surface and MME segmentation of multiple sclerosis subjects from retinal OCT

Optical coherence tomography (OCT) is a noninvasive imaging modality that can be used to obtain depth images of the retina. Patients with multiple sclerosis (MS) have thinning retinal nerve fiber and ganglion cell layers, and approximately 5% of MS patients will develop microcystic macular edema (MME) within the retina. Segmentation of both the retinal layers and MME can provide important information to help monitor MS progression. Graph-based segmentation with machine learning preprocessing is the leading method for retinal layer segmentation, providing accurate surface delineations with the correct topological ordering. However, graph methods are time-consuming and they do not optimally incorporate joint MME segmentation. This paper presents a deep network that extracts continuous, smooth, and topology-guaranteed surfaces and MMEs. The network learns shape priors automatically during training rather than being hard-coded as in graph methods. In this new approach, retinal surfaces and MMEs are segmented together with two cascaded deep networks in a single feed forward propagation. The proposed framework obtains retinal surfaces (separating the layers) with sub-pixel surface accuracy comparable to the best existing graph methods and MMEs with better accuracy than the state-of-the-art method. The full segmentation operation takes only ten seconds for a 3D volume. © 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

[1]  J. Fujimoto,et al.  Optical coherence tomography of the human retina. , 1995, Archives of ophthalmology.

[2]  Xiaodong Wu,et al.  Automated 3-D Intraretinal Layer Segmentation of Macular Spectral-Domain Optical Coherence Tomography Images , 2009, IEEE Transactions on Medical Imaging.

[3]  F. Medeiros,et al.  Detection of glaucoma progression with stratus OCT retinal nerve fiber layer, optic nerve head, and macular thickness measurements. , 2009, Investigative ophthalmology & visual science.

[4]  Joseph A. Izatt,et al.  Automatic segmentation of seven retinal layers in SDOCT images congruent with expert manual segmentation , 2010, Optics express.

[5]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[6]  S. A. Meyer,et al.  Visual dysfunction in multiple sclerosis correlates better with optical coherence tomography derived estimates of macular ganglion cell layer thickness than peripapillary retinal nerve fiber layer thickness , 2011, Multiple sclerosis.

[7]  S. A. Meyer,et al.  Primary retinal pathology in multiple sclerosis as detected by optical coherence tomography. , 2011, Brain : a journal of neurology.

[8]  C. Crainiceanu,et al.  Microcystic macular oedema, thickness of the inner nuclear layer of the retina, and disease characteristics in multiple sclerosis: a retrospective study , 2012, The Lancet Neurology.

[9]  A. Green,et al.  Microcystic macular oedema in multiple sclerosis is associated with disease severity. , 2012, Brain : a journal of neurology.

[10]  Tara N. Sainath,et al.  Improving deep neural networks for LVCSR using rectified linear units and dropout , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[11]  Jerry L Prince,et al.  Retinal layer segmentation of macular OCT images using boundary classification , 2013, Biomedical optics express.

[12]  S. A. Meyer,et al.  Active MS is associated with accelerated retinal ganglion cell/inner plexiform layer thinning , 2013, Neurology.

[13]  Aaron Carass,et al.  Multiple-object geometric deformable model for segmentation of macular OCT. , 2014, Biomedical optics express.

[14]  J. Álvarez-cermeño,et al.  Comparative Diagnostic Accuracy of Ganglion Cell-Inner Plexiform and Retinal Nerve Fiber Layer Thickness Measures by Cirrus and Spectralis Optical Coherence Tomography in Relapsing-Remitting Multiple Sclerosis , 2014, BioMed research international.

[15]  Peter A. Calabresi,et al.  Microcystic macular edema detection in retina OCT images , 2014, Medical Imaging.

[16]  Jerry L Prince,et al.  Automatic segmentation of microcystic macular edema in OCT. , 2014, Biomedical optics express.

[17]  Jerry L Prince,et al.  Applying an Open-Source Segmentation Algorithm to Different OCT Devices in Multiple Sclerosis Patients and Healthy Controls: Implications for Clinical Trials , 2015, Multiple sclerosis international.

[18]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[19]  Jelena Novosel,et al.  Loosely coupled level sets for simultaneous 3D retinal layer segmentation in optical coherence tomography , 2015, Medical Image Anal..

[20]  Vibhav Vineet,et al.  Conditional Random Fields as Recurrent Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[21]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Ramiro S. Maldonado,et al.  The application of optical coherence tomography in neurologic diseases , 2015, Neurology. Clinical practice.

[23]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Jerry L Prince,et al.  An Overview of the Multi-Object Geometric Deformable Model Approach in Biomedical Imaging , 2016 .

[25]  Jing Tian,et al.  Performance evaluation of automated segmentation software on optical coherence tomography volume data , 2016, Journal of biophotonics.

[26]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[27]  M. Mühlau,et al.  Retinal inner nuclear layer volume reflects response to immunotherapy in multiple sclerosis. , 2016, Brain : a journal of neurology.

[28]  Ghassan Hamarneh,et al.  Topology Aware Fully Convolutional Networks for Histology Gland Segmentation , 2016, MICCAI.

[29]  Hariharan Ravishankar,et al.  Learning and Incorporating Shape Models for Semantic Segmentation , 2017, MICCAI.

[30]  Yue Wu,et al.  Deep-Learning Based, Automated Segmentation of Macular Edema in Optical Coherence Tomography , 2017, bioRxiv.

[31]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Alain Trouvé,et al.  Atlas‐based shape analysis and classification of retinal optical coherence tomography images using the functional shape (fshape) framework , 2017, Medical Image Anal..

[33]  Nassir Navab,et al.  ReLayNet: Retinal Layer and Fluid Segmentation of Macular Optical Coherence Tomography using Fully Convolutional Network , 2017, Biomedical optics express.

[34]  Thomas Theelen,et al.  Robust total retina thickness segmentation in optical coherence tomography images using convolutional neural networks. , 2017, Biomedical optics express.

[35]  Aaron Carass,et al.  Improving graph-based OCT segmentation for severe pathology in retinitis pigmentosa patients , 2017, Medical Imaging.

[36]  Hao Chen,et al.  3D deeply supervised network for automated segmentation of volumetric medical images , 2017, Medical Image Anal..

[37]  Peter A. Calabresi,et al.  Towards Topological Correct Segmentation of Macular OCT from Cascaded FCNs , 2017, FIFI/OMIA@MICCAI.

[38]  Chong Wang,et al.  Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search. , 2017, Biomedical optics express.

[39]  Bianca S. Gerendas,et al.  Fully Automated Detection and Quantification of Macular Fluid in OCT Using Deep Learning. , 2017, Ophthalmology.

[40]  Peter A. Calabresi,et al.  Multi-layer fast level set segmentation for macular OCT , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[41]  Peter A. Calabresi,et al.  Topology guaranteed segmentation of the human retina from OCT using convolutional neural networks , 2018, ArXiv.

[42]  Leixin Zhou,et al.  Multiple surface segmentation using convolution neural nets: application to retinal layer segmentation in OCT images , 2018, Biomedical optics express.

[43]  Jeny Rajan,et al.  A benchmark study of automated intra-retinal cyst segmentation algorithms using optical coherence tomography B-scans , 2018, Comput. Methods Programs Biomed..

[44]  Eliza M. Gordon-Lipkin,et al.  Retinal measurements predict 10‐year disability in multiple sclerosis , 2019, Annals of clinical and translational neurology.

[45]  Aaron Carass,et al.  Cerebellum parcellation with convolutional neural networks , 2019, Medical Imaging: Image Processing.

[46]  Peter A. Calabresi,et al.  Retinal layer parcellation of optical coherence tomography images: Data resource for multiple sclerosis and healthy controls , 2018, Data in brief.