Automated Neuron Reconstruction from 3D Fluorescence Microscopy Images Using Sequential Monte Carlo Estimation

Microscopic images of neuronal cells provide essential structural information about the key constituents of the brain and form the basis of many neuroscientific studies. Computational analyses of the morphological properties of the captured neurons require first converting the structural information into digital tree-like reconstructions. Many dedicated computational methods and corresponding software tools have been and are continuously being developed with the aim to automate this step while achieving human-comparable reconstruction accuracy. This pursuit is hampered by the immense diversity and intricacy of neuronal morphologies as well as the often low quality and ambiguity of the images. Here we present a novel method we developed in an effort to improve the robustness of digital reconstruction against these complicating factors. The method is based on probabilistic filtering by sequential Monte Carlo estimation and uses prediction and update models designed specifically for tracing neuronal branches in microscopic image stacks. Moreover, it uses multiple probabilistic traces to arrive at a more robust, ensemble reconstruction. The proposed method was evaluated on fluorescence microscopy image stacks of single neurons and dense neuronal networks with expert manual annotations serving as the gold standard, as well as on synthetic images with known ground truth. The results indicate that our method performs well under varying experimental conditions and compares favorably to state-of-the-art alternative methods.

[1]  Qingming Luo,et al.  Automated and Accurate Detection of Soma Location and Surface Morphology in Large-Scale 3D Neuron Images , 2013, PloS one.

[2]  Andrew R. Cohen,et al.  Automated tracing and volume measurements of neurons from 3‐D confocal fluorescence microscopy data , 1994, Journal of microscopy.

[3]  R. C Cannon,et al.  An on-line archive of reconstructed hippocampal neurons , 1998, Journal of Neuroscience Methods.

[4]  Erik H. W. Meijering,et al.  Automated neuron tracing using probability hypothesis density filtering , 2017, Bioinform..

[5]  Pascal Fua,et al.  Delineating trees in noisy 2D images and 3D image-stacks , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Ghassan Hamarneh,et al.  Vascular bifurcation detection in scale-space , 2012, 2012 IEEE Workshop on Mathematical Methods in Biomedical Image Analysis.

[7]  Eugene W. Myers,et al.  Automatic 3D neuron tracing using all-path pruning , 2011, Bioinform..

[8]  Giorgio A. Ascoli,et al.  Digital Reconstructions of Neuronal Morphology: Three Decades of Research Trends , 2012, Front. Neurosci..

[9]  Daniel Racoceanu,et al.  Reconstructing neuronal morphology from microscopy stacks using fast marching , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[10]  Nando de Freitas,et al.  An Introduction to Sequential Monte Carlo Methods , 2001, Sequential Monte Carlo Methods in Practice.

[11]  Hanchuan Peng,et al.  Automatic tracing of ultra-volumes of neuronal images , 2016, Nature Methods.

[12]  Steve M. Potter,et al.  MDL Constrained 3-D Grayscale Skeletonization Algorithm for Automated Extraction of Dendrites and Spines from Fluorescence Confocal Images , 2009, Neuroinformatics.

[13]  Michael Scholz,et al.  New methods for the computer-assisted 3-D reconstruction of neurons from confocal image stacks , 2004, NeuroImage.

[14]  Stephen L. Senft,et al.  A Brief History of Neuronal Reconstruction , 2011, Neuroinformatics.

[15]  Hanchuan Peng,et al.  V3D enables real-time 3D visualization and quantitative analysis of large-scale biological image data sets , 2010, Nature Biotechnology.

[16]  Ju Lu,et al.  The DIADEM Data Sets: Representative Light Microscopy Images of Neuronal Morphology to Advance Automation of Digital Reconstructions , 2011, Neuroinformatics.

[17]  Tianming Liu,et al.  SmartTracing: self-learning-based Neuron reconstruction , 2015, Brain Informatics.

[18]  Badrinath Roysam,et al.  Improved detection of branching points in algorithms for automated neuron tracing from 3D confocal images , 2008, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[19]  Julio Chapeton,et al.  Active learning of neuron morphology for accurate automated tracing of neurites , 2014, Front. Neuroanat..

[20]  G. Kitagawa Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[21]  Chia-Ling Tsai,et al.  A Broadly Applicable 3-D Neuron Tracing Method Based on Open-Curve Snake , 2011, Neuroinformatics.

[22]  Alejandro F. Frangi,et al.  Muliscale Vessel Enhancement Filtering , 1998, MICCAI.

[23]  Giorgio A. Ascoli,et al.  Automated reconstruction of neuronal morphology: An overview , 2011, Brain Research Reviews.

[24]  Pavel V. Belichenko,et al.  Confocal Laser Scanning Microscopy and 3-D Reconstructions of Neuronal Structures in Human Brain Cortex , 1995, NeuroImage.

[25]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[26]  Giorgio A. Ascoli,et al.  Trees of the Brain, Roots of the Mind , 2015 .

[27]  Alexander Borst,et al.  One Rule to Grow Them All: A General Theory of Neuronal Branching and Its Practical Application , 2010, PLoS Comput. Biol..

[28]  Hang Zhou,et al.  NeuroGPS-Tree: automatic reconstruction of large-scale neuronal populations with dense neurites , 2015, Nature Methods.

[29]  Erik Meijering,et al.  Neuron tracing in perspective , 2010, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[30]  E M Glaser,et al.  Neuron imaging with Neurolucida--a PC-based system for image combining microscopy. , 1990, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[31]  Erik H. W. Meijering,et al.  Automated neuron morphology reconstruction using fuzzy-logic detection and Bayesian tracing algorithms , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[32]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[33]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Badrinath Roysam,et al.  3-D Image Pre-processing Algorithms for Improved Automated Tracing of Neuronal Arbors , 2011, Neuroinformatics.

[35]  Vincent Lepetit,et al.  Multiscale Centerline Detection , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[37]  Ioannis A. Kakadiaris,et al.  Automatic Morphological Reconstruction of Neurons from Multiphoton and Confocal Microscopy Images Using 3D Tubular Models , 2015, Neuroinformatics.

[38]  G. Ascoli,et al.  NeuroMorpho.Org: A Central Resource for Neuronal Morphologies , 2007, The Journal of Neuroscience.

[39]  Scott T. Acton,et al.  Tubularity Flow Field—A Technique for Automatic Neuron Segmentation , 2015, IEEE Transactions on Image Processing.

[40]  Hans-Christian Hege,et al.  The Filament Editor: An Interactive Software Environment for Visualization, Proof-Editing and Analysis of 3D Neuron Morphology , 2013, Neuroinformatics.

[41]  Hanchuan Peng,et al.  A distance-field based automatic neuron tracing method , 2013, BMC Bioinformatics.

[42]  Gongning Luo,et al.  Neuron anatomy structure reconstruction based on a sliding filter , 2015, BMC Bioinformatics.

[43]  Roberto Marcondes Cesar Junior,et al.  Automatic contour extraction from 2D neuron images , 2009, Journal of Neuroscience Methods.

[44]  Karel Svoboda,et al.  The Past, Present, and Future of Single Neuron Reconstruction , 2011, Neuroinformatics.

[45]  Zhi Zhou,et al.  Adaptive Image Enhancement for Tracing 3D Morphologies of Neurons and Brain Vasculatures , 2014, Neuroinformatics.

[46]  Eugene W. Myers,et al.  Automated Reconstruction of Neuronal Morphology Based on Local Geometrical and Global Structural Models , 2011, Neuroinformatics.

[47]  H. V. Wheal,et al.  A system for quantitative morphological measurement and electrotonic modelling of neurons: three-dimensional reconstruction , 1993, Journal of Neuroscience Methods.

[48]  Pascal Fua,et al.  Reconstructing Loopy Curvilinear Structures Using Integer Programming , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Xiaobo Zhou,et al.  Automated neurite labeling and analysis in fluorescence microscopy images , 2006, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[50]  J J Capowski,et al.  Accurate computer reconstruction and graphics display of complex neurons utilizing state-of-the-art interactive techniques. , 1981, Computers and biomedical research, an international journal.

[51]  Ioannis A. Kakadiaris,et al.  Improved Automatic Centerline Tracing for Dendritic and Axonal Structures , 2015, Neuroinformatics.

[52]  Sean L. Hill,et al.  BigNeuron: Large-Scale 3D Neuron Reconstruction from Optical Microscopy Images , 2015, Neuron.

[53]  Shaoqun Zeng,et al.  Rapid Reconstruction of 3D Neuronal Morphology from Light Microscopy Images with Augmented Rayburst Sampling , 2013, PloS one.

[54]  Badrinath Roysam,et al.  The FARSIGHT Trace Editor: An Open Source Tool for 3-D Inspection and Efficient Pattern Analysis Aided Editing of Automated Neuronal Reconstructions , 2011, Neuroinformatics.

[55]  G. Ascoli Computational Neuroanatomy , 2002, Humana Press.

[56]  Giorgio A. Ascoli,et al.  A Comparative Computer Simulation of Dendritic Morphology , 2008, PLoS Comput. Biol..

[57]  Hanchuan Peng,et al.  From DIADEM to BigNeuron , 2015, Neuroinformatics.

[58]  Sidong Liu,et al.  Rivulet: 3D Neuron Morphology Tracing with Iterative Back-Tracking , 2016, Neuroinformatics.

[59]  Giulio Iannello,et al.  Automated Neuron Tracing Methods: An Updated Account , 2016, Neuroinformatics.

[60]  Shuiwang Ji,et al.  Deep Learning Segmentation of Optical Microscopy Images Improves 3-D Neuron Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[61]  J. Douglas Armstrong,et al.  Bioinformatics Applications Note Systems Biology Simple Neurite Tracer: Open Source Software for Reconstruction, Visualization and Analysis of Neuronal Processes , 2022 .

[62]  Eugene W. Myers,et al.  Proof-editing is the Bottleneck Of 3D Neuron Reconstruction: The Problem and Solutions , 2011, Neuroinformatics.

[63]  Hanchuan Peng,et al.  APP2: automatic tracing of 3D neuron morphology based on hierarchical pruning of a gray-weighted image distance-tree , 2013, Bioinform..

[64]  E M GLASER,et al.  A SEMI-AUTOMATIC COMPUTER-MICROSCOPE FOR THE ANALYSIS OF NEURONAL MORPHOLOGY. , 1965, IEEE transactions on bio-medical engineering.

[65]  Erik H. W. Meijering,et al.  Fuzzy-Logic Based Detection and Characterization of Junctions and Terminations in Fluorescence Microscopy Images of Neurons , 2015, Neuroinformatics.

[66]  Shih-Fu Chang,et al.  Automatic Reconstruction of Neural Morphologies with Multi-Scale Tracking , 2012, Front. Neural Circuits.

[67]  E Meijering,et al.  Design and validation of a tool for neurite tracing and analysis in fluorescence microscopy images , 2004, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[68]  Wei Tsang Ooi,et al.  Neurite Tracing With Object Process , 2016, IEEE Transactions on Medical Imaging.

[69]  Jun S. Liu,et al.  Sequential Imputations and Bayesian Missing Data Problems , 1994 .

[70]  M Masseroli,et al.  Quantitative morphology and shape classification of neurons by computerized image analysis. , 1993, Computer methods and programs in biomedicine.

[71]  Hanchuan Peng,et al.  Virtual finger boosts three-dimensional imaging and microsurgery as well as terabyte volume image visualization and analysis , 2014, Nature Communications.

[72]  Hang Zhou,et al.  NeuroGPS: automated localization of neurons for brain circuits using L1 minimization model , 2013, Scientific Reports.

[73]  Kevin W Eliceiri,et al.  NIH Image to ImageJ: 25 years of image analysis , 2012, Nature Methods.

[74]  Pascal Fua,et al.  Automated Reconstruction of Dendritic and Axonal Trees by Global Optimization with Geometric Priors , 2011, Neuroinformatics.

[75]  Brian R. Tietz,et al.  Deciding Which Way to Go: How Do Insects Alter Movements to Negotiate Barriers? , 2012, Front. Neurosci..

[76]  Zhi Zhou,et al.  TReMAP: Automatic 3D Neuron Reconstruction Based on Tracing, Reverse Mapping and Assembling of 2D Projections , 2015, Neuroinformatics.

[77]  Changming Sun,et al.  Junction detection for linear structures based on Hessian, correlation and shape information , 2012, Pattern Recognit..

[78]  Scott T. Acton,et al.  Vector field convolution medialness applied to neuron tracing , 2013, 2013 IEEE International Conference on Image Processing.

[79]  Hanchuan Peng,et al.  Extensible visualization and analysis for multidimensional images using Vaa3D , 2014, Nature Protocols.

[80]  Yuan Liu,et al.  DIADEMchallenge.Org: A Compendium of Resources Fostering the Continuous Development of Automated Neuronal Reconstruction , 2011, Neuroinformatics.

[81]  Colin J. R. Sheppard,et al.  Signal-to-Noise Ratio in Confocal Microscopes , 2006 .

[82]  Erik H. W. Meijering,et al.  Neuron reconstruction from fluorescence microscopy images using sequential Monte Carlo estimation , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).