Deep convolutional neural networks for segmenting 3D in vivo multiphoton images of vasculature in Alzheimer disease mouse models

The health and function of tissue rely on its vasculature network to provide reliable blood perfusion. Volumetric imaging approaches, such as multiphoton microscopy, are able to generate detailed 3D images of blood vessels that could contribute to our understanding of the role of vascular structure in normal physiology and in disease mechanisms. The segmentation of vessels, a core image analysis problem, is a bottleneck that has prevented the systematic comparison of 3D vascular architecture across experimental populations. We explored the use of convolutional neural networks to segment 3D vessels within volumetric in vivo images acquired by multiphoton microscopy. We evaluated different network architectures and machine learning techniques in the context of this segmentation problem. We show that our optimized convolutional neural network architecture with a customized loss function, which we call DeepVess, yielded a segmentation accuracy that was better than state-of-the-art methods, while also being orders of magnitude faster than the manual annotation. To explore the effects of aging and Alzheimer’s disease on capillaries, we applied DeepVess to 3D images of cortical blood vessels in young and old mouse models of Alzheimer’s disease and wild type littermates. We found little difference in the distribution of capillary diameter or tortuosity between these groups, but did note a decrease in the number of longer capillary segments (>75μm) in aged animals as compared to young, in both wild type and Alzheimer’s disease mouse models.

[1]  D. Kleinfeld,et al.  Two-Photon Microscopy as a Tool to Study Blood Flow and Neurovascular Coupling in the Rodent Brain , 2013 .

[2]  D. Kleinfeld,et al.  The cortical angiome: an interconnected vascular network with noncolumnar patterns of blood flow , 2013, Nature Neuroscience.

[3]  Thomas Krucker,et al.  Computer-based analysis of microvascular alterations in a mouse model for Alzheimer's disease , 2007, SPIE Medical Imaging.

[4]  B R Masters,et al.  Two-photon excitation fluorescence microscopy. , 2000, Annual review of biomedical engineering.

[5]  Rangasami L. Kashyap,et al.  Building Skeleton Models via 3-D Medial Surface/Axis Thinning Algorithms , 1994, CVGIP Graph. Model. Image Process..

[6]  Michael D. Abràmoff,et al.  Image processing with ImageJ , 2004 .

[7]  Luc Van Gool,et al.  Deep Retinal Image Understanding , 2016, MICCAI.

[8]  David A Boas,et al.  Spatio-temporal dynamics of cerebral capillary segments with stalling red blood cells , 2017, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[9]  Claus Wolff-Menzler,et al.  Brain Aging: Models, Methods and Mechanisms, R. David, Riddle (Eds.). CRC Press, Taylor & Francis Group (2007), price: £ 85.00, ISBN: 978-0-8493-3818-2 , 2009 .

[10]  K. Hossmann Viability thresholds and the penumbra of focal ischemia , 1994, Annals of neurology.

[12]  Mingchen Gao,et al.  Deep vessel tracking: A generalized probabilistic approach via deep learning , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[13]  D. Kleinfeld,et al.  Correlations of Neuronal and Microvascular Densities in Murine Cortex Revealed by Direct Counting and Colocalization of Nuclei and Vessels , 2009, The Journal of Neuroscience.

[14]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[17]  R N Kalaria,et al.  Cerebral vessels in ageing and Alzheimer's disease. , 1996, Pharmacology & therapeutics.

[18]  P. So,et al.  Two-Photon deep tissue ex vivo imaging of mouse dermal and subcutaneous structures. , 1998, Optics express.

[19]  Philipp Schneider,et al.  Hierarchical microimaging for multiscale analysis of large vascular networks , 2006, NeuroImage.

[20]  David R. Riddle,et al.  Microvascular plasticity in aging , 2003, Ageing Research Reviews.

[21]  David R. Riddle,et al.  Regulation of Cerebrovascular Aging , 2007 .

[22]  Isabelle Bloch,et al.  A review of 3D vessel lumen segmentation techniques: Models, features and extraction schemes , 2009, Medical Image Anal..

[23]  Laurent D. Cohen,et al.  Deformable tree models for 2D and 3D branching structures extraction , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[24]  Joanna L. Jankowsky,et al.  Mutant presenilins specifically elevate the levels of the 42 residue β-amyloid peptide in vivo: evidence for augmentation of a 42-specific γ secretase , 2004 .

[25]  T A Woolsey,et al.  Increased brain capillaries in chronic hypoxia. , 1999, Journal of applied physiology.

[26]  Thomas Krucker,et al.  Altered morphology and 3D architecture of brain vasculature in a mouse model for Alzheimer's disease , 2008, Proceedings of the National Academy of Sciences.

[27]  F. Cassot,et al.  Tortuosity and other vessel attributes for arterioles and venules of the human cerebral cortex. , 2014, Microvascular research.

[28]  Huazhu Fu,et al.  Retinal vessel segmentation via deep learning network and fully-connected conditional random fields , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[29]  Bojana Stefanovic,et al.  Amyloid-β-dependent compromise of microvascular structure and function in a model of Alzheimer's disease. , 2012, Brain : a journal of neurology.

[30]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[31]  K. Fujita [Two-photon laser scanning fluorescence microscopy]. , 2007, Tanpakushitsu kakusan koso. Protein, nucleic acid, enzyme.

[32]  D. Kleinfeld,et al.  All-Optical Histology Using Ultrashort Laser Pulses , 2003, Neuron.

[33]  Dallas E. Johnson,et al.  Analysis of Messy Data Volume 1: Designed Experiments, Second Edition , 2004 .

[34]  D. Kleinfeld,et al.  Fluctuating and sensory-induced vasodynamics in rodent cortex extend arteriole capacity , 2011, Proceedings of the National Academy of Sciences.

[35]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[36]  Stephen Lin,et al.  DeepVessel: Retinal Vessel Segmentation via Deep Learning and Conditional Random Field , 2016, MICCAI.

[37]  Karel Svoboda,et al.  ScanImage: Flexible software for operating laser scanning microscopes , 2003, Biomedical engineering online.

[38]  Francis K. H. Quek,et al.  A review of vessel extraction techniques and algorithms , 2004, CSUR.

[39]  Kullervo Hynynen,et al.  Deep Learning Convolutional Networks for Multiphoton Microscopy Vasculature Segmentation , 2016, ArXiv.

[40]  Nozomi Nishimura,et al.  In Vivo Calcium Imaging of Cardiomyocytes in the Beating Mouse Heart With Multiphoton Microscopy , 2018, Front. Physiol..

[41]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[42]  Bunyarit Uyyanonvara,et al.  Blood vessel segmentation methodologies in retinal images - A survey , 2012, Comput. Methods Programs Biomed..

[43]  Kim Mouridsen,et al.  Effect of electrical forepaw stimulation on capillary transit-time heterogeneity (CTH) , 2016, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[44]  Ruth J. Muschel,et al.  Extracting 3D Vascular Structures from Microscopy Images using Convolutional Recurrent Networks , 2017, ArXiv.

[45]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[46]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[47]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[48]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[49]  Ralph Müller,et al.  Novel three-dimensional analysis tool for vascular trees indicates complete micro-networks, not single capillaries, as the angiogenic endpoint in mice overexpressing human VEGF165 in the brain , 2008, NeuroImage.

[50]  Jong Beom Ra,et al.  A locally adaptive region growing algorithm for vascular segmentation , 2003, Int. J. Imaging Syst. Technol..

[51]  Jun Zou,et al.  High‐speed photoacoustic microscopy of mouse cortical microhemodynamics , 2017, Journal of biophotonics.

[52]  Robert H. Cudmore,et al.  Cerebral vascular structure in the motor cortex of adult mice is stable and is not altered by voluntary exercise , 2017, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[53]  Myriam Peyrounette,et al.  Neutrophil adhesion in brain capillaries contributes to cortical blood flow decreases and impaired memory function in a mouse model of Alzheimer’s disease , 2017, bioRxiv.

[54]  H. Reinhold,et al.  A QUANTITATIVE STUDY OF AGE‐RELATED CHANGES IN THE VASCULAR ARCHITECTURE OF THE RAT CEREBRAL CORTEX , 1981, Neuropathology and applied neurobiology.

[55]  M. M. Fraza,et al.  Blood vessel segmentation methodologies in retinal images – A survey , 2015 .

[56]  Chantal Rémy,et al.  A Direct Method for Measuring Mouse Capillary Cortical Blood Volume Using Multiphoton Laser Scanning Microscopy , 2007, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[57]  Julia A. Schnabel,et al.  Segmentation of Vasculature From Fluorescently Labeled Endothelial Cells in Multi-Photon Microscopy Images , 2019, IEEE Transactions on Medical Imaging.

[58]  Jean-Philippe Thirion,et al.  Image matching as a diffusion process: an analogy with Maxwell's demons , 1998, Medical Image Anal..

[59]  D. Attwell,et al.  Capillary pericytes regulate cerebral blood flow in health and disease , 2014, Nature.

[60]  C. Iadecola Neurovascular regulation in the normal brain and in Alzheimer's disease , 2004, Nature Reviews Neuroscience.

[61]  Tom Vercauteren,et al.  Diffeomorphic demons: Efficient non-parametric image registration , 2009, NeuroImage.

[62]  Myriam Peyrounette,et al.  Neutrophil adhesion in brain capillaries reduces cortical blood flow and impairs memory function in Alzheimer’s disease mouse models , 2018, Nature Neuroscience.

[63]  Bojana Stefanovic,et al.  Venular degeneration leads to vascular dysfunction in a transgenic model of Alzheimer's disease. , 2015, Brain : a journal of neurology.

[64]  K. Svoboda,et al.  Long-term, high-resolution imaging in the mouse neocortex through a chronic cranial window , 2009, Nature Protocols.

[65]  Bruce J Tromberg,et al.  Imaging coronary artery microstructure using second-harmonic and two-photon fluorescence microscopy. , 2004, Biophysical journal.

[66]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[67]  Alberto Bravin,et al.  In vivo two-photon microscopy study of short-term effects of microbeam irradiation on normal mouse brain microvasculature. , 2006, International journal of radiation oncology, biology, physics.

[68]  Philipp Schneider,et al.  Hierarchical bioimaging and quantification of vasculature in disease models using corrosion casts and microcomputed tomography , 2004, SPIE Optics + Photonics.

[69]  P. Luiten,et al.  Cerebral microvascular pathology in aging and Alzheimer's disease , 2001, Progress in Neurobiology.

[70]  Anil K. Jain,et al.  A modified Hausdorff distance for object matching , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[71]  O. Hunziker,et al.  The aging human cerebral cortex: a stereological characterization of changes in the capillary net. , 1979, Journal of gerontology.