Probabilistic spatial analysis in quantitative microscopy with uncertainty-aware cell detection using deep Bayesian regression

The investigation of biological systems with three-dimensional microscopy demands automatic cell identification methods that not only are accurate but also can imply the uncertainty in their predictions. The use of deep learning to regress density maps is a popular successful approach for extracting cell coordinates from local peaks in a postprocessing step, which then, however, hinders any meaningful probabilistic output. We propose a framework that can operate on large microscopy images and output probabilistic predictions (i) by integrating deep Bayesian learning for the regression of uncertainty-aware density maps, where peak detection algorithms generate cell proposals, and (ii) by learning a mapping from prediction proposals to a probabilistic space that accurately represents the chances of a successful prediction. Using these calibrated predictions, we propose a probabilistic spatial analysis with Monte Carlo sampling. We demonstrate this in a bone marrow dataset, where our proposed methods reveal spatial patterns that are otherwise undetectable.

[1]  M. Unser,et al.  DeepImageJ: A user-friendly environment to run deep learning models in ImageJ , 2021, Nature Methods.

[2]  Alvaro Gomariz,et al.  Utilizing Uncertainty Estimation in Deep Learning Segmentation of Fluorescence Microscopy Images with Missing Markers , 2021, 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI).

[3]  Yan Huang,et al.  Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Marius Pachitariu,et al.  Cellpose: a generalist algorithm for cellular segmentation , 2020, Nature Methods.

[5]  Mark A. Anastasio,et al.  Deeply-Supervised Density Regression for Automatic Cell Counting in Microscopy Images , 2020, Medical Image Anal..

[6]  Orcun Goksel,et al.  Modality Attention and Sampling Enables Deep Learning with Heterogeneous Marker Combinations in Fluorescence Microscopy , 2020, Nature Machine Intelligence.

[7]  Matthias Bo Stuart,et al.  Detection and Localization of Ultrasound Scatterers Using Convolutional Neural Networks , 2020, IEEE Transactions on Medical Imaging.

[8]  Yuxin Zheng,et al.  Manifold-Regularized Regression Network: A Novel End-to-End Method for Cell Counting and Localization , 2020, ICIAI.

[9]  Andrés G. Marrugo,et al.  Generating density maps for convolutional neural network-based cell counting in specular microscopy images , 2020, Journal of Physics: Conference Series.

[10]  Patrick M. Helbling,et al.  Imaging and spatial analysis of hematopoietic stem cell niches , 2020, Annals of the New York Academy of Sciences.

[11]  Marius Pachitariu,et al.  Cellpose: a generalist algorithm for cellular segmentation , 2020, Nature Methods.

[12]  Purang Abolmaesumi,et al.  Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation , 2019, IEEE Transactions on Medical Imaging.

[13]  Balaji Lakshminarayanan,et al.  Deep Ensembles: A Loss Landscape Perspective , 2019, ArXiv.

[14]  Qian Liu,et al.  Automated Counting of Cancer Cells by Ensembling Deep Features , 2019, Cells.

[15]  Jianfeng Gao,et al.  On the Variance of the Adaptive Learning Rate and Beyond , 2019, ICLR.

[16]  Kaiyong Zhao,et al.  AutoML: A Survey of the State-of-the-Art , 2019, Knowl. Based Syst..

[17]  Yuta Hiasa,et al.  Automated Muscle Segmentation from Clinical CT Using Bayesian U-Net for Personalized Musculoskeletal Modeling , 2019, IEEE Transactions on Medical Imaging.

[18]  Christine Tanner,et al.  Siamese Networks With Location Prior for Landmark Tracking in Liver Ultrasound Sequences , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[19]  Thomas Brox,et al.  U-Net: deep learning for cell counting, detection, and morphometry , 2018, Nature Methods.

[20]  Alejandro F. Frangi,et al.  Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 , 2018, Lecture Notes in Computer Science.

[21]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Patrick M. Helbling,et al.  Quantitative spatial analysis of haematopoiesis-regulating stromal cells in the bone marrow microenvironment by 3D microscopy , 2018, Nature Communications.

[23]  Eugene W. Myers,et al.  Cell Detection with Star-convex Polygons , 2018, MICCAI.

[24]  Andrew Zisserman,et al.  Microscopy cell counting and detection with fully convolutional regression networks , 2018, Comput. methods Biomech. Biomed. Eng. Imaging Vis..

[25]  Joseph Redmon,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[26]  Hai Su,et al.  Efficient and robust cell detection: A structured regression approach , 2018, Medical Image Anal..

[27]  Loic A. Royer,et al.  Content-aware image restoration: pushing the limits of fluorescence microscopy , 2018, bioRxiv.

[28]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[29]  M. Manz,et al.  Quantification and three-dimensional microanatomical organization of the bone marrow. , 2017, Blood advances.

[30]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Erik Meijering,et al.  Imagining the future of bioimage analysis , 2016, Nature Biotechnology.

[32]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[33]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[34]  Zhiyu Zhao,et al.  Deep imaging of bone marrow shows non-dividing stem cells are mainly perisinusoidal , 2015, Nature.

[35]  Andrew Y. Ng,et al.  End-to-End People Detection in Crowded Scenes , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  B. S. Manjunath,et al.  Characterizing spatial distributions of astrocytes in the mammalian retina , 2015, Bioinform..

[37]  Xiaogang Wang,et al.  Cross-scene crowd counting via deep convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[40]  Xi C. He,et al.  Megakaryocytes maintain homeostatic quiescence and promote post-injury regeneration of hematopoietic stem cells , 2014, Nature Medicine.

[41]  P. Frenette,et al.  Hematopoietic stem cell niche maintenance during homeostasis and regeneration , 2014, Nature Medicine.

[42]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Jean-Christophe Olivo-Marin,et al.  Analysis of the Spatial Organization of Molecules with Robust Statistics , 2013, PloS one.

[44]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Erlend Hodneland,et al.  CellSegm - a MATLAB toolbox for high-throughput 3D cell segmentation , 2013, Source Code for Biology and Medicine.

[46]  A. Protopopov,et al.  Quantitative imaging of haematopoietic stem and progenitor cell localization and hypoxic status in the bone marrow microenvironment , 2013, Nature Cell Biology.

[47]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[48]  Deborah S. Barkauskas,et al.  Dynamic Imaging of Marrow-Resident Granulocytes Interacting with Human Mesenchymal Stem Cells upon Systemic Lipopolysaccharide Challenge , 2013, Stem cells international.

[49]  Johannes E. Schindelin,et al.  Fiji: an open-source platform for biological-image analysis , 2012, Nature Methods.

[50]  D. Scadden,et al.  Rethinking stroma: lessons from the blood. , 2012, Cell stem cell.

[51]  Ullrich Köthe,et al.  Ilastik: Interactive learning and segmentation toolkit , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[52]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[53]  Jichun Chen,et al.  Hematopoiesis in 3 dimensions: human and murine bone marrow architecture visualized by confocal microscopy. , 2010, Blood.

[54]  Ivo F. Sbalzarini,et al.  Beyond co-localization: inferring spatial interactions between sub-cellular structures from microscopy images , 2010, BMC Bioinformatics.

[55]  Xiaobo Zhou,et al.  Integrating multi-scale blob/curvilinear detector techniques and multi-level sets for automated segmentation of stem cell images , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[56]  Anne E Carpenter,et al.  CellProfiler Analyst: data exploration and analysis software for complex image-based screens , 2008, BMC Bioinformatics.

[57]  A. Raftery,et al.  Strictly Proper Scoring Rules, Prediction, and Estimation , 2007 .

[58]  S. Morrison,et al.  Supplemental Experimental Procedures , 2022 .

[59]  C Wählby,et al.  Combining intensity, edge and shape information for 2D and 3D segmentation of cell nuclei in tissue sections , 2004, Journal of microscopy.

[60]  L. Breiman Random Forests , 2001, Machine Learning.

[61]  J J Vaquero,et al.  Applying watershed algorithms to the segmentation of clustered nuclei. , 1998, Cytometry.

[62]  Tony Lindeberg,et al.  Feature Detection with Automatic Scale Selection , 1998, International Journal of Computer Vision.

[63]  D. W. Scott Multivariate Density Estimation: Theory, Practice, and Visualization , 1992, Wiley Series in Probability and Statistics.

[64]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[65]  Carmen Domingo Biology , 1929, Nature.

[66]  I. Sbalzarini Seeing is believing – quantifying is convincing: Computational image analysis in biology ∗ , 2015 .

[67]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[68]  D. Link,et al.  Cellular Complexity of the Bone Marrow Hematopoietic Stem Cell Niche , 2013, Calcified Tissue International.

[69]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[70]  A. Gelfand,et al.  Spatial Point Patterns , 2010 .

[71]  Xiaobo Zhou,et al.  Automatic Segmentation of High-Throughput RNAi Fluorescent Cellular Images , 2008, IEEE Transactions on Information Technology in Biomedicine.

[72]  G. Brier VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[73]  L. Zon,et al.  Hematopoiesis: An Evolving Paradigm for Stem Cell Biology , 2008, Cell.