Anatomical modeling of brain vasculature in two-photon microscopy by generalizable deep learning

Objective and Impact Statement Segmentation of blood vessels from two-photon microscopy (2PM) angiograms of brains has important applications in hemodynamic analysis and disease diagnosis. Here we develop a generalizable deep learning technique for accurate 2PM vascular segmentation of sizable regions in mouse brains acquired from multiple 2PM setups. The technique is computationally efficient, thus ideal for large-scale neurovascular analysis. Introduction Vascular segmentation from 2PM angiograms is an important first step in hemodynamic modeling of brain vasculature. Existing segmentation methods based on deep learning either lack the ability to generalize to data from different imaging systems, or are computationally infeasible for large-scale angiograms. In this work, we overcome both these limitations by a method that is generalizable to various imaging systems, and is able to segment large-scale angiograms. Methods We employ a computationally efficient deep learning framework with a loss function that incorporates a balanced binary-cross-entropy loss and a total variation regularization on the network’s output. Its effectiveness is demonstrated on experimentally acquired in-vivo angiograms from mouse brains of dimensions up to 808×808×702 μm. Results To demonstrate the superior generalizability of our framework, we train on data from only one 2PM microscope, and demonstrate high-quality segmentation on data from a different microscope without any network tuning. Overall, our method demonstrates 10× faster computation in terms of voxels-segmented-per-second and 3× larger depth compared to the state-of-the-art. Conclusion Our work provides a generalizable and computationally efficient anatomical modeling framework for brain vasculature, which consists of deep learning based vascular segmentation followed by graphing. It paves the way for future modeling and analysis of hemodynamic response at much greater scales that were inaccessible before.

[1]  Mert R. Sabuncu,et al.  Deep convolutional neural networks for segmenting 3D in vivo multiphoton images of vasculature in Alzheimer disease mouse models , 2018, PloS one.

[2]  Konstantinos Kamnitsas,et al.  Unsupervised domain adaptation in brain lesion segmentation with adversarial networks , 2016, IPMI.

[3]  A. A. Aqrawi,et al.  Detecting Salt Domes Using a Dip Guided 3D Sobel Seismic Attribute , 2011 .

[4]  Prabir Kumar Biswas,et al.  Deep neural ensemble for retinal vessel segmentation in fundus images towards achieving label-free angiography , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[5]  Sabri Boughorbel,et al.  Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric , 2017, PloS one.

[6]  S. Charpak,et al.  Imaging local neuronal activity by monitoring PO2 transients in capillaries , 2013, Nature Medicine.

[7]  William M. Wells,et al.  Medical Image Computing and Computer-Assisted Intervention — MICCAI’98 , 1998, Lecture Notes in Computer Science.

[8]  Klaas Paul Pruessmann,et al.  Probing neuronal activation by functional quantitative susceptibility mapping under a visual paradigm: A group level comparison with BOLD fMRI and PET , 2016, NeuroImage.

[9]  Anna Devor,et al.  Dependence of the MR signal on the magnetic susceptibility of blood studied with models based on real microvascular networks , 2019, Magnetic resonance in medicine.

[10]  Ghassan Hamarneh,et al.  Live-Vessel: Extending Livewire for Simultaneous Extraction of Optimal Medial and Boundary Paths in Vascular Images , 2007, MICCAI.

[11]  Lior Wolf,et al.  Unsupervised Microvascular Image Segmentation Using an Active Contours Mimicking Neural Network , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  Allan Hanbury,et al.  Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool , 2015, BMC Medical Imaging.

[13]  Anna Devor,et al.  Modeling of Cerebral Oxygen Transport Based on In vivo Microscopic Imaging of Microvascular Network Structure, Blood Flow, and Oxygenation , 2016, Front. Comput. Neurosci..

[14]  Kaleem Siddiqi,et al.  Flux Maximizing Geometric Flows , 2001, ICCV.

[15]  Bostjan Likar,et al.  Enhancement of Vascular Structures in 3D and 2D Angiographic Images , 2016, IEEE Transactions on Medical Imaging.

[16]  Ting Liu,et al.  Unsupervised Total Variation Loss for Semi-supervised Deep Learning of Semantic Segmentation , 2016, ArXiv.

[17]  Kullervo Hynynen,et al.  Deep Learning Convolutional Networks for Multiphoton Microscopy Vasculature Segmentation , 2016, ArXiv.

[18]  Xiaogang Wang,et al.  Deep Self-Learning From Noisy Labels , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[19]  Yoshua Bengio,et al.  Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach , 2011, ICML.

[20]  José Manuel Bravo,et al.  A Function for Quality Evaluation of Retinal Vessel Segmentations , 2012, IEEE Transactions on Medical Imaging.

[21]  Nico Karssemeijer,et al.  Transfer Learning for Domain Adaptation in MRI: Application in Brain Lesion Segmentation , 2017, MICCAI.

[22]  Farida Cheriet,et al.  Automatic Graph-Based Modeling of Brain Microvessels Captured With Two-Photon Microscopy , 2019, IEEE Journal of Biomedical and Health Informatics.

[23]  Qiang Chen,et al.  Robust noise region-based active contour model via local similarity factor for image segmentation , 2017, Pattern Recognit..

[24]  Anna Devor,et al.  Validation and optimization of hypercapnic-calibrated fMRI from oxygen-sensitive two-photon microscopy , 2016, Philosophical Transactions of the Royal Society B: Biological Sciences.

[25]  Min Bai,et al.  Learning Deep Structured Active Contours End-to-End , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Lei Tian,et al.  Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media , 2018, Optica.

[27]  Yoshua Bengio,et al.  The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[28]  Anders M. Dale,et al.  A vascular anatomical network model of the spatio-temporal response to brain activation , 2008, NeuroImage.

[29]  Farida Cheriet,et al.  Laplacian Flow Dynamics on Geometric Graphs for Anatomical Modeling of Cerebrovascular Networks , 2019, IEEE Transactions on Medical Imaging.

[30]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[31]  Gideon S. Mann,et al.  Generalized Expectation Criteria for Semi-Supervised Learning with Weakly Labeled Data , 2010, J. Mach. Learn. Res..

[32]  Klaus Scheffler,et al.  The impact of vessel size, orientation and intravascular contribution on the neurovascular fingerprint of BOLD bSSFP fMRI , 2017, NeuroImage.

[33]  Axel Montagne,et al.  Cerebral blood flow regulation and neurovascular dysfunction in Alzheimer disease , 2017, Nature Reviews Neuroscience.

[34]  Bjoern H. Menze,et al.  DeepVesselNet: Vessel Segmentation, Centerline Prediction, and Bifurcation Detection in 3-D Angiographic Volumes , 2018, Frontiers in Neuroscience.

[35]  W. Denk,et al.  Deep tissue two-photon microscopy , 2005, Nature Methods.

[36]  Alejandro F. Frangi,et al.  Muliscale Vessel Enhancement Filtering , 1998, MICCAI.

[37]  Nassir Navab,et al.  Medical Image Computing and Computer-Assisted Intervention – MICCAI 2013 , 2013, Lecture Notes in Computer Science.

[38]  Nassir Navab,et al.  Deep Active Contours , 2016, ArXiv.

[39]  Anna Devor,et al.  Quantifying the Microvascular Origin of BOLD-fMRI from First Principles with Two-Photon Microscopy and an Oxygen-Sensitive Nanoprobe , 2015, The Journal of Neuroscience.

[40]  Emiri T. Mandeville,et al.  Two-photon high-resolution measurement of partial pressure of oxygen in cerebral vasculature and tissue , 2010, Nature Methods.

[41]  Max W. K. Law,et al.  Three Dimensional Curvilinear Structure Detection Using Optimally Oriented Flux , 2008, ECCV.

[42]  Sotirios A. Tsaftaris,et al.  Medical Image Computing and Computer Assisted Intervention , 2017 .

[43]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).