Neuronal Population Reconstruction From Ultra-Scale Optical Microscopy Images via Progressive Learning

Reconstruction of neuronal populations from ultra-scale optical microscopy (OM) images is essential to investigate neuronal circuits and brain mechanisms. The noises, low contrast, huge memory requirement, and high computational cost pose significant challenges in the neuronal population reconstruction. Recently, many studies have been conducted to extract neuron signals using deep neural networks (DNNs). However, training such DNNs usually relies on a huge amount of voxel-wise annotations in OM images, which are expensive in terms of both finance and labor. In this paper, we propose a novel framework for dense neuronal population reconstruction from ultra-scale images. To solve the problem of high cost in obtaining manual annotations for training DNNs, we propose a progressive learning scheme for neuronal population reconstruction (PLNPR) which does not require any manual annotations. Our PLNPR scheme consists of a traditional neuron tracing module and a deep segmentation network that mutually complement and progressively promote each other. To reconstruct dense neuronal populations from a terabyte-sized ultra-scale image, we introduce an automatic framework which adaptively traces neurons block by block and fuses fragmented neurites in overlapped regions continuously and smoothly. We build a dataset “VISoR-40” which consists of 40 large-scale OM image blocks from cortical regions of a mouse. Extensive experimental results on our VISoR-40 dataset and the public BigNeuron dataset demonstrate the effectiveness and superiority of our method on neuronal population reconstruction and single neuron reconstruction. Furthermore, we successfully apply our method to reconstruct dense neuronal populations from an ultra-scale mouse brain slice. The proposed adaptive block propagation and fusion strategies greatly improve the completeness of neurites in dense neuronal population reconstruction.

[1]  Ioannis A. Kakadiaris,et al.  Automatic Morphological Reconstruction of Neurons from Multiphoton and Confocal Microscopy Images Using 3D Tubular Models , 2015, Neuroinformatics.

[2]  Chia-Ling Tsai,et al.  A Broadly Applicable 3-D Neuron Tracing Method Based on Open-Curve Snake , 2011, Neuroinformatics.

[3]  Qingming Luo,et al.  3D BrainCV: Simultaneous visualization and analysis of cells and capillaries in a whole mouse brain with one-micron voxel resolution , 2014, NeuroImage.

[4]  Kei Ito,et al.  Efficient Monte Carlo Image Analysis for the Location of Vascular Entity , 2015, IEEE Transactions on Medical Imaging.

[5]  Pascal Fua,et al.  Tracing in 2D to Reduce the Annotation Effort for 3D Deep Delineation , 2018, Medical Image Anal..

[6]  Shin Ishii,et al.  PAT—Probabilistic Axon Tracking for Densely Labeled Neurons in Large 3-D Micrographs , 2019, IEEE Transactions on Medical Imaging.

[7]  Hanchuan Peng,et al.  Automatic reconstruction of 3D neuron structures using a graph-augmented deformable model , 2010, Bioinform..

[8]  R. C Cannon,et al.  An on-line archive of reconstructed hippocampal neurons , 1998, Journal of Neuroscience Methods.

[9]  Tianming Liu,et al.  SmartTracing: self-learning-based Neuron reconstruction , 2015, Brain Informatics.

[10]  Hanchuan Peng,et al.  Automatic tracing of ultra-volumes of neuronal images , 2016, Nature Methods.

[11]  Xiaoyang Liu,et al.  FMST: an Automatic Neuron Tracing Method Based on Fast Marching and Minimum Spanning Tree , 2018, Neuroinformatics.

[12]  N. De Stefano,et al.  Clinical use of brain volumetry , 2013, Journal of magnetic resonance imaging : JMRI.

[13]  Sidong Liu,et al.  Rivulet: 3D Neuron Morphology Tracing with Iterative Back-Tracking , 2016, Neuroinformatics.

[14]  Zhi Zhou,et al.  TReMAP: Automatic 3D Neuron Reconstruction Based on Tracing, Reverse Mapping and Assembling of 2D Projections , 2015, Neuroinformatics.

[15]  Hanchuan Peng,et al.  A Multiscale Ray-Shooting Model for Termination Detection of Tree-Like Structures in Biomedical Images , 2019, IEEE Transactions on Medical Imaging.

[16]  Zhiwei Xiong,et al.  Scalable volumetric imaging for ultrahigh-speed brain mapping at synaptic resolution , 2017, bioRxiv.

[17]  Stephen L. Senft,et al.  A Brief History of Neuronal Reconstruction , 2011, Neuroinformatics.

[18]  R. Coleman,et al.  Neuroimaging and early diagnosis of Alzheimer disease: a look to the future. , 2003, Radiology.

[19]  Kibok Lee,et al.  Robust Inference via Generative Classifiers for Handling Noisy Labels , 2019, ICML.

[20]  Hang Zhou,et al.  NeuroGPS-Tree: automatic reconstruction of large-scale neuronal populations with dense neurites , 2015, Nature Methods.

[21]  Deniz Erdogmus,et al.  Principal Curves as Skeletons of Tubular Objects , 2011, Neuroinformatics.

[22]  Paolo Frasconi,et al.  Large-scale automated identification of mouse brain cells in confocal light sheet microscopy images , 2014, Bioinform..

[23]  Hanchuan Peng,et al.  A distance-field based automatic neuron tracing method , 2013, BMC Bioinformatics.

[24]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[25]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[26]  Zhi Zhou,et al.  DeepNeuron: an open deep learning toolbox for neuron tracing , 2018, Brain Informatics.

[27]  Eugene W. Myers,et al.  Unsupervised segmentation of noisy electron microscopy images using salient watersheds and region merging , 2013, BMC Bioinformatics.

[28]  Eugene W. Myers,et al.  Automated Reconstruction of Neuronal Morphology Based on Local Geometrical and Global Structural Models , 2011, Neuroinformatics.

[29]  Weidong Cai,et al.  Automated 3D Neuron Tracing with Precise Branch Erasing and Confidence Controlled Back-Tracking , 2017, bioRxiv.

[30]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[31]  Dong Liu,et al.  Deep High-Resolution Representation Learning for Human Pose Estimation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Wei Tsang Ooi,et al.  Neurite Tracing With Object Process , 2016, IEEE Transactions on Medical Imaging.

[33]  Zhi Zhou,et al.  Ensemble Neuron Tracer for 3D Neuron Reconstruction , 2017, Neuroinformatics.

[34]  Eugene W. Myers,et al.  Automatic 3D neuron tracing using all-path pruning , 2011, Bioinform..

[35]  Feng Lin,et al.  A Graph-Theoretical Approach for Tracing Filamentary Structures in Neuronal and Retinal Images , 2016, IEEE Transactions on Medical Imaging.

[36]  Jinhyun Kim,et al.  neuTube 1.0: A New Design for Efficient Neuron Reconstruction Software Based on the SWC Format 123 , 2015, eNeuro.

[37]  Hanchuan Peng,et al.  Virtual finger boosts three-dimensional imaging and microsurgery as well as terabyte volume image visualization and analysis , 2014, Nature Communications.

[38]  Stephen T. C. Wong,et al.  Repulsive force based snake model to segment and track neuronal axons in 3D microscopy image stacks , 2006, NeuroImage.

[39]  Hang Zhou,et al.  NeuroGPS: automated localization of neurons for brain circuits using L1 minimization model , 2013, Scientific Reports.

[40]  Bo Zhang,et al.  Neuron Segmentation Based on CNN with Semi-Supervised Regularization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[41]  John P. Cunningham,et al.  Automated scalable segmentation of neurons from multispectral images , 2016, NIPS.

[42]  Shuiwang Ji,et al.  Deep Learning Segmentation of Optical Microscopy Images Improves 3-D Neuron Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[43]  Zhi Zhou,et al.  Neuron crawler: An automatic tracing algorithm for very large neuron images , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[44]  Hanchuan Peng,et al.  APP2: automatic tracing of 3D neuron morphology based on hierarchical pruning of a gray-weighted image distance-tree , 2013, Bioinform..

[45]  Erik H. W. Meijering,et al.  Automated neuron tracing using probability hypothesis density filtering , 2017, Bioinform..

[46]  Huiqi Li,et al.  Segment 2D and 3D Filaments by Learning Structured and Contextual Features , 2017, IEEE Transactions on Medical Imaging.

[47]  Yali Wang,et al.  MetaCleaner: Learning to Hallucinate Clean Representations for Noisy-Labeled Visual Recognition , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Simon K. Warfield,et al.  Deep learning with noisy labels: exploring techniques and remedies in medical image analysis , 2020, Medical Image Anal..

[49]  Hao Chen,et al.  Automatic 3D Cardiovascular MR Segmentation with Densely-Connected Volumetric ConvNets , 2017, MICCAI.

[50]  Hao Chen,et al.  3D deeply supervised network for automated segmentation of volumetric medical images , 2017, Medical Image Anal..

[51]  Sean L. Hill,et al.  BigNeuron: Large-Scale 3D Neuron Reconstruction from Optical Microscopy Images , 2015, Neuron.

[52]  Heng Wang,et al.  Memory and Time Efficient 3D Neuron Morphology Tracing in Large-Scale Images , 2018, 2018 Digital Image Computing: Techniques and Applications (DICTA).

[53]  Dong Liu,et al.  Progressive Learning for Neuronal Population Reconstruction from Optical Microscopy Images , 2019, MICCAI.