PointNeuron: 3D Neuron Reconstruction via Geometry and Topology Learning of Point Clouds

Digital neuron reconstruction from 3D microscopy images is an essential technique for investigating brain connectomics and neuron morphology. Existing reconstruction frameworks use convolution-based segmentation networks to partition the neuron from noisy backgrounds before applying the tracing algorithm. The tracing results are sensitive to the raw image quality and segmentation accuracy. In this paper, we propose a novel framework for 3D neuron reconstruction. Our key idea is to use the geometric representation power of the point cloud to better explore the intrinsic structural information of neurons. Our proposed framework adopts one graph convolutional network to predict the neural skeleton points and another one to produce the connectivity of these points. We finally generate the target SWC file through the interpretation of the predicted point coordinates, radius, and connections. Evaluated on the Janelia-Fly dataset from the BigNeuron project, we show that our framework achieves competitive neuron reconstruction performance. Our geometry and topology learning of point clouds could further benefit 3D medical image analysis, such as cardiac surface reconstruction. Our code is available at https://github.com/RunkaiZhao/PointNeuron.

[1]  Lixu Gu,et al.  Spine Segmentation with Multi-view GCN and Boundary Constraint , 2022, 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC).

[2]  Sean L. Hill,et al.  BigNeuron: A resource to benchmark and predict best-performing algorithms for automated reconstruction of neuronal morphology , 2022, bioRxiv.

[3]  Binh-Son Hua,et al.  Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segmentation , 2022, MICCAI.

[4]  Tiange Xiang,et al.  3D Medical Point Transformer: Introducing Convolution to Attention Networks for Medical Point Cloud Analysis , 2021, ArXiv.

[5]  Siqi Liu,et al.  AI-Enhanced 3D Biomedical Data Analytics for Neuronal Structure Reconstruction , 2021, Humanity Driven AI.

[6]  Alejandro F Frangi,et al.  Shape registration with learned deformations for 3D shape reconstruction from sparse and incomplete point clouds , 2021, Medical Image Anal..

[7]  Heng Wang,et al.  Voxel-wise Cross-Volume Representation Learning for 3D Neuron Reconstruction , 2021, MLMI@MICCAI.

[8]  Chaoyi Zhang,et al.  Single Neuron Segmentation Using Graph-Based Global Reasoning with Auxiliary Skeleton Loss from 3D Optical Microscope Images , 2021, 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI).

[9]  Yi-King Choi,et al.  Point2Skeleton: Learning Skeletal Representations from Point Clouds , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Yang Wang,et al.  Learning Hybrid Representations for Automatic 3D Vessel Centerline Extraction , 2020, MICCAI.

[11]  Bostjan Likar,et al.  Vascular Surface Segmentation for Intracranial Aneurysm Isolation and Quantification , 2020, MICCAI.

[12]  Yang Song,et al.  3D Conditional Adversarial Learning for Synthesizing Microscopic Neuron Image Using Skeleton-to-Neuron Translation , 2020, 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).

[13]  Jonathan Masci,et al.  Tractogram filtering of anatomically non-plausible fibers with geometric deep learning , 2020, MICCAI.

[14]  Linlin Shen,et al.  3D Neuron Reconstruction in Tangled Neuronal Image With Deep Networks , 2020, IEEE Transactions on Medical Imaging.

[15]  Olivier Scheidegger,et al.  Learning Shape Representation on Sparse Point Clouds for Volumetric Image Segmentation , 2019, MICCAI.

[16]  Mei Chen,et al.  Multiscale Kernels for Enhanced U-Shaped Network to Improve 3D Neuron Tracing , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[17]  Yunhong Wang,et al.  Adaptive NMS: Refining Pedestrian Detection in a Crowd , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Yue Wang,et al.  Segmenting Neuronal Structure in 3D Optical Microscope Images via Knowledge Distillation with Teacher-Student Network , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[19]  Heng Wang,et al.  Memory and Time Efficient 3D Neuron Morphology Tracing in Large-Scale Images , 2018, 2018 Digital Image Computing: Techniques and Applications (DICTA).

[20]  Xiaoyang Liu,et al.  FMST: an Automatic Neuron Tracing Method Based on Fast Marching and Minimum Spanning Tree , 2018, Neuroinformatics.

[21]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[22]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[23]  Larry S. Davis,et al.  Soft-NMS — Improving Object Detection with One Line of Code , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Shuiwang Ji,et al.  Deep Learning Segmentation of Optical Microscopy Images Improves 3-D Neuron Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[25]  Weidong Cai,et al.  Automated 3D Neuron Tracing with Precise Branch Erasing and Confidence Controlled Back-Tracking , 2017, bioRxiv.

[26]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[27]  Hao Chen,et al.  3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes , 2016, MICCAI.

[28]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[29]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[30]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Sidong Liu,et al.  Reconstruction of 3D neuron morphology using Rivulet back-tracking , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[32]  Yaozong Gao,et al.  Deformable MR Prostate Segmentation via Deep Feature Learning and Sparse Patch Matching , 2016, IEEE Transactions on Medical Imaging.

[33]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.

[34]  Li Cheng,et al.  Learning to Boost Filamentary Structure Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[35]  Hanchuan Peng,et al.  SmartTracing: self-learning-based Neuron reconstruction , 2015, Brain Informatics.

[36]  Sean L. Hill,et al.  BigNeuron: Large-Scale 3D Neuron Reconstruction from Optical Microscopy Images , 2015, Neuron.

[37]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[38]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[39]  Jinhyun Kim,et al.  neuTube 1.0: A New Design for Efficient Neuron Reconstruction Software Based on the SWC Format 123 , 2015, eNeuro.

[40]  Henggui Zhang,et al.  A Pipeline for Neuron Reconstruction Based on Spatial Sliding Volume Filter Seeding , 2014, Comput. Math. Methods Medicine.

[41]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Julio Chapeton,et al.  Active learning of neuron morphology for accurate automated tracing of neurites , 2014, Front. Neuroanat..

[43]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Pascal Fua,et al.  Reconstructing Loopy Curvilinear Structures Using Integer Programming , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Hanchuan Peng,et al.  APP2: automatic tracing of 3D neuron morphology based on hierarchical pruning of a gray-weighted image distance-tree , 2013, Bioinform..

[46]  Ann-Shyn Chiang,et al.  High-throughput Computer Method for 3D Neuronal Structure Reconstruction from the Image Stack of the Drosophila Brain and Its Applications , 2012, PLoS Comput. Biol..

[47]  Shih-Fu Chang,et al.  Automatic Reconstruction of Neural Morphologies with Multi-Scale Tracking , 2012, Front. Neural Circuits.

[48]  Slawomir J. Nasuto,et al.  Neuromantic – from Semi-Manual to Semi-Automatic Reconstruction of Neuron Morphology , 2012, Front. Neuroinform..

[49]  E. Myers,et al.  Automatic 3D neuron tracing using all-path pruning , 2011, Bioinform..

[50]  Eugene W. Myers,et al.  Automated Reconstruction of Neuronal Morphology Based on Local Geometrical and Global Structural Models , 2011, Neuroinformatics.

[51]  Chia-Ling Tsai,et al.  A Broadly Applicable 3-D Neuron Tracing Method Based on Open-Curve Snake , 2011, Neuroinformatics.

[52]  Deniz Erdogmus,et al.  Principal Curves as Skeletons of Tubular Objects , 2011, Neuroinformatics.

[53]  Ju Lu,et al.  The DIADEM Data Sets: Representative Light Microscopy Images of Neuronal Morphology to Advance Automation of Digital Reconstructions , 2011, Neuroinformatics.

[54]  Hanchuan Peng,et al.  Automatic reconstruction of 3D neuron structures using a graph-augmented deformable model , 2010, Bioinform..

[55]  R. W. Draft,et al.  Transgenic strategies for combinatorial expression of fluorescent proteins in the nervous system , 2007, Nature.

[56]  Graham Harman.  Network , 2021, Future Theory.