CapillaryNet: An Automated System to Analyze Microcirculation Videos from Handheld Vital Microscopy

Capillaries are the smallest vessels in the body responsible for the delivery of oxygen and nutrients to the surrounding cells. Various diseases have been shown to alter the density of nutritive capillaries and the flow velocity of erythrocytes. In previous studies, capillary density and flow velocity have been assessed manually by trained specialists. Manual analysis of a 20-second long microvascular video takes on average 20 minutes and requires extensive training. Several studies have reported that manual analysis hinders the application of microvascular microscopy in a clinical setting. In this paper, we present a fully automated system, called CapillaryNet, that can automate microvascular microscopy analysis and thus enable the method to be used not just as a research tool, but also for clinical applications. Our method has been developed by acquiring microcirculation videos from 50 different subjects annotated by trained biomedical researchers. CapillaryNet detects capillaries with an accuracy comparable to trained researchers in less than 0.1% of the time taken by humans and measures several microvascular parameters that researchers were previously unable to quantify, i.e. capillary hematocrit and intra-capillary flow velocity heterogeneity.

[1]  C. Ince,et al.  Microcirculation: Physiology, Pathophysiology, and Clinical Application , 2019, Blood Purification.

[2]  Ferdinand van der Heijden,et al.  Efficient adaptive density estimation per image pixel for the task of background subtraction , 2006, Pattern Recognit. Lett..

[3]  T. Wester,et al.  Skin microcirculation in healthy term newborn infants--assessment of morphology, perfusion and oxygenation. , 2015, Clinical hemorheology and microcirculation.

[4]  Robert Ritch,et al.  Peripapillary perfused capillary density in primary open-angle glaucoma across disease stage: an optical coherence tomography angiography study , 2017, British Journal of Ophthalmology.

[5]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[6]  C. Michel,et al.  Starling: the formulation of his hypothesis of microvascular fluid exchange and its significance after 100 years , 1997, Experimental physiology.

[7]  Z. Zivkovic.  Improved adaptive Gaussian mixture model for background subtraction , 2004, ICPR 2004.

[8]  Mikhail V Volkov,et al.  Video capillaroscopy clarifies mechanism of the photoplethysmographic waveform appearance , 2017, Scientific Reports.

[9]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[10]  C. Ince,et al.  Quantitative assessment of the microcirculation in healthy volunteers and in patients with septic shock* , 2012, Critical care medicine.

[11]  Haibin Ling,et al.  Salient Object Detection in the Deep Learning Era: An In-Depth Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Jean-Charles Preiser,et al.  Microvascular blood flow is altered in patients with sepsis. , 2002, American journal of respiratory and critical care medicine.

[13]  A. Shore,et al.  Variability in Sublingual Microvessel Density and Flow Measurements in Healthy Volunteers , 2009, Microcirculation.

[14]  I. Manyonda,et al.  Is capillary rarefaction an independent risk marker for cardiovascular disease in South Asians? , 2011, Journal of Human Hypertension.

[15]  Alireza Nasiri Avanaki.  Exact Histogram Specification Optimized for Structural Similarity , 2009, ArXiv.

[16]  Thomas G. Dietterich,et al.  Principal Curvature-Based Region Detector for Object Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Kayvan Najarian,et al.  An automated method for analysis of microcirculation videos for accurate assessment of tissue perfusion , 2012, BMC Medical Imaging.

[18]  Z. A. Awan,et al.  Skin microvascular morphology and hemodynamics during treatment with veno-arterial extra-corporeal membrane oxygenation. , 2014, Clinical hemorheology and microcirculation.

[19]  M Intaglietta,et al.  Microcirculation: its significance in clinical and molecular medicine , 1997, Journal of internal medicine.

[20]  Johannes G. G. Dobbe,et al.  Med Biol Eng Comput (2011) 49:1269–1278 DOI 10.1007/s11517-011-0824-1 ORIGINAL ARTICLE , 2022 .

[21]  S. Lagarde,et al.  Influence of peripheral arterial disease on capillary pressure in the foot. , 2003, Journal of vascular surgery.

[22]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[23]  C. Ince,et al.  MicroTools enables automated quantification of capillary density and red blood cell velocity in handheld vital microscopy , 2019, Communications Biology.

[24]  Maiken Nedergaard,et al.  Erythrocytes Are Oxygen-Sensing Regulators of the Cerebral Microcirculation , 2016, Neuron.

[25]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[26]  Ali Borji,et al.  Salient object detection: A survey , 2014, Computational Visual Media.

[27]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[28]  T. The,et al.  Decreased nailfold capillary density in Raynaud's phenomenon: a reflection of immunologically mediated local and systemic vascular disease? , 1985, Annals of the rheumatic diseases.

[29]  W. Kraus,et al.  Capillary density of skeletal muscle: a contributing mechanism for exercise intolerance in class II-III chronic heart failure independent of other peripheral alterations. , 1999, Journal of the American College of Cardiology.

[30]  Johannes G. G. Dobbe,et al.  Measurement of functional microcirculatory geometry and velocity distributions using automated image analysis , 2008, Medical & Biological Engineering & Computing.

[31]  C. Ince,et al.  Increasing arterial blood pressure with norepinephrine does not improve microcirculatory blood flow: a prospective study , 2009, Critical care.

[32]  Nao Sasaki,et al.  Capillary rarefaction: an early marker of microvascular disease in young hemodialysis patients , 2014, Clinical kidney journal.

[33]  Keiichi Abe,et al.  Topological structural analysis of digitized binary images by border following , 1985, Comput. Vis. Graph. Image Process..

[34]  Alireza Nasiri Avanaki,et al.  Exact global histogram specification optimized for structural similarity , 2008, 0901.0065.

[35]  J. Parrillo,et al.  Point-of-care assessment of microvascular blood flow in critically ill patients , 2009, Intensive Care Medicine.

[36]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[37]  J. Bakker,et al.  Second consensus on the assessment of sublingual microcirculation in critically ill patients: results from a task force of the European Society of Intensive Care Medicine , 2018, Intensive Care Medicine.

[38]  Frédéric Lesage,et al.  Compromised microvascular oxygen delivery increases brain tissue vulnerability with age , 2018, Scientific Reports.

[39]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  C. Ince,et al.  Effects of early hemodynamic resuscitation on left ventricular performance and microcirculatory function during endotoxic shock , 2015, Intensive Care Medicine Experimental.

[41]  K. Kvernebo,et al.  Noninvasive assessments of oxygen delivery from the microcirculation to skin in hypothermia-treated asphyxiated newborn infants , 2016, Pediatric Research.

[42]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[43]  A C Shore,et al.  Capillaroscopy and the measurement of capillary pressure. , 2000, British journal of clinical pharmacology.

[44]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[45]  Jason D. Allen,et al.  Relationship between leg muscle capillary density and peak hyperemic blood flow with endurance capacity in peripheral artery disease. , 2011, Journal of applied physiology.

[46]  R. Schneider,et al.  Nailfold capillary density is importantly associated over time with muscle and skin disease activity in juvenile dermatomyositis. , 2011, Rheumatology.

[47]  N. McGill,et al.  The Established and Evolving Role of Nailfold Capillaroscopy in Connective-Tissue Disease , 2018, Connective Tissue Disease - Current State of the Art.

[48]  C. Ince,et al.  Assessment of endothelial cell function and physiological microcirculatory reserve by video microscopy using a topical acetylcholine and nitroglycerin challenge , 2017, Intensive Care Medicine Experimental.

[49]  C. Stehouwer,et al.  Assessing Microvascular Function in Humans from a Chronic Disease Perspective. , 2017, Journal of the American Society of Nephrology : JASN.

[50]  Austin Roorda,et al.  Noninvasive visualization and analysis of parafoveal capillaries in humans. , 2010, Investigative ophthalmology & visual science.

[51]  Ferdinand van der Heijden,et al.  Recursive unsupervised learning of finite mixture models , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.