Learning Dense Voxel Embeddings for 3D Neuron Reconstruction

We show dense voxel embeddings learned via deep metric learning can be employed to produce a highly accurate segmentation of neurons from 3D electron microscopy images. A metric graph on an arbitrary set of short and long-range edges can be constructed from the dense embeddings generated by a convolutional network. Partitioning the metric graph with long-range affinities as repulsive constraints can produce an initial segmentation with high precision, with substantial improvements on very thin objects. The convolutional embedding net is reused without any modification to agglomerate the systematic splits caused by complex "self-touching"' objects. Our proposed method achieves state-of-the-art accuracy on the challenging problem of 3D neuron reconstruction from the brain images acquired by serial section electron microscopy. Our alternative, object-centered representation could be more generally useful for other computational tasks in automated neural circuit reconstruction.

[1]  Joseph F. Murray,et al.  Convolutional Networks Can Learn to Generate Affinity Graphs for Image Segmentation , 2010, Neural Computation.

[2]  Jianbo Shi,et al.  Machine Learning of Hierarchical Clustering to Segment 2D and 3D Images , 2013, PloS one.

[3]  Nir Shavit,et al.  The big data challenges of connectomics , 2014, Nature Neuroscience.

[4]  H. Sebastian Seung,et al.  Image Segmentation by Size-Dependent Single Linkage Clustering of a Watershed Basin Graph , 2015, ArXiv.

[5]  William R. Gray Roncal,et al.  Saturated Reconstruction of a Volume of Neocortex , 2015, Cell.

[6]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[7]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[8]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[9]  Iasonas Kokkinos,et al.  Learning Dense Convolutional Embeddings for Semantic Segmentation , 2015, ArXiv.

[10]  Joachim M. Buhmann,et al.  Crowdsourcing the creation of image segmentation algorithms for connectomics , 2015, Front. Neuroanat..

[11]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[12]  Peter Li,et al.  Flood-Filling Networks , 2016, ArXiv.

[13]  Pieter Abbeel,et al.  Combinatorial Energy Learning for Image Segmentation , 2015, NIPS.

[14]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[15]  Hayk Saribekyan,et al.  A Multicore Path to Connectomics-on-Demand , 2017, PPoPP 2017.

[16]  Luc Van Gool,et al.  Semantic Instance Segmentation with a Discriminative Loss Function , 2017, ArXiv.

[17]  H. Sebastian Seung,et al.  An Error Detection and Correction Framework for Connectomics , 2017, NIPS.

[18]  Peter H. Li,et al.  High-Precision Automated Reconstruction of Neurons with Flood-filling Networks , 2017 .

[19]  H. Sebastian Seung,et al.  Superhuman Accuracy on the SNEMI3D Connectomics Challenge , 2017, ArXiv.

[20]  Peng Wang,et al.  Semantic Instance Segmentation via Deep Metric Learning , 2017, ArXiv.

[21]  Sashank J. Reddi,et al.  On the Convergence of Adam and Beyond , 2018, ICLR.

[22]  Ullrich Köthe,et al.  The Mutex Watershed: Efficient, Parameter-Free Image Partitioning , 2018, ECCV.

[23]  Shu Kong,et al.  Recurrent Pixel Embedding for Instance Grouping , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Lei Zheng,et al.  Instance Segmentation of Fibers from Low Resolution CT Scans via 3D Deep Embedding Learning , 2019, BMVC.

[25]  Ming Yang,et al.  Multi-Lane Detection via Multi-Task Network in Various Road Scenes , 2018, 2018 Chinese Automation Congress (CAC).

[26]  Thomas Neff,et al.  Instance Segmentation and Tracking with Cosine Embeddings and Recurrent Hourglass Networks , 2018, MICCAI.

[27]  Zaïd Harchaoui,et al.  Object Discovery in Videos as Foreground Motion Clustering , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Bernard Ghanem,et al.  3D Instance Segmentation via Multi-Task Metric Learning , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[29]  Chandan Singh,et al.  Large Scale Image Segmentation with Structured Loss Based Deep Learning for Connectome Reconstruction , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  H. Sebastian Seung,et al.  Learning Metric Graphs for Neuron Segmentation in Electron Microscopy Images , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[31]  Bastian Leibe,et al.  3D-BEVIS: Birds-Eye-View Instance Segmentation , 2019, GCPR.

[32]  Xiaoyong Shen,et al.  Learning Shape-Aware Embedding for Scene Text Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Ming Yang,et al.  3D Graph Embedding Learning with a Structure-aware Loss Function for Point Cloud Semantic Instance Segmentation , 2019, ArXiv.

[34]  Fred A. Hamprecht,et al.  A Generalized Framework for Agglomerative Clustering of Signed Graphs applied to Instance Segmentation , 2019, ArXiv.

[35]  Rahil Garnavi,et al.  Deep Semantic Instance Segmentation of Tree-Like Structures Using Synthetic Data , 2018, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[36]  Zihan Zhou,et al.  Single-Image Piece-Wise Planar 3D Reconstruction via Associative Embedding , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Duc Thanh Nguyen,et al.  JSIS3D: Joint Semantic-Instance Segmentation of 3D Point Clouds With Multi-Task Pointwise Networks and Multi-Value Conditional Random Fields , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).