Leveraging Domain Knowledge to improve EM image segmentation with Lifted Multicuts

The throughput of electron microscopes has increased significantly in recent years, enabling detailed analysis of cell morphology and ultrastructure. Analysis of neural circuits at single-synapse resolution remains the flagship target of this technique, but applications to cell and developmental biology are also starting to emerge at scale. The amount of data acquired in such studies makes manual instance segmentation, a fundamental step in many analysis pipelines, impossible. While automatic segmentation approaches have improved significantly thanks to the adoption of convolutional neural networks, their accuracy still lags behind human annotations and requires additional manual proof-reading. A major hindrance to further improvements is the limited field of view of the segmentation networks preventing them from exploiting the expected cell morphology or other prior biological knowledge which humans use to inform their segmentation decisions. In this contribution, we show how such domain-specific information can be leveraged by expressing it as long-range interactions in a graph partitioning problem known as the lifted multicut problem. Using this formulation, we demonstrate significant improvement in segmentation accuracy for three challenging EM segmentation problems from neuroscience and cell biology.

[1]  Chandan Singh,et al.  Large Scale Image Segmentation with Structured Loss Based Deep Learning for Connectome Reconstruction , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Ullrich Köthe,et al.  Globally Optimal Closed-Surface Segmentation for Connectomics , 2012, ECCV.

[3]  Ullrich Köthe,et al.  The Mutex Watershed and its Objective: Efficient, Parameter-Free Image Partitioning , 2019, ArXiv.

[4]  G. Knott,et al.  Ultrastructurally-smooth thick partitioning and volume stitching for larger-scale connectomics , 2015, Nature Methods.

[5]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[6]  Fred A. Hamprecht,et al.  Automated Detection of Synapses in Serial Section Transmission Electron Microscopy Image Stacks , 2014, PloS one.

[7]  Eric T. Trautman,et al.  A Complete Electron Microscopy Volume of the Brain of Adult Drosophila melanogaster , 2017, Cell.

[8]  Thomas Brox,et al.  Joint Graph Decomposition & Node Labeling: Problem, Algorithms, Applications , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  H. Sebastian Seung,et al.  Superhuman Accuracy on the SNEMI3D Connectomics Challenge , 2017, ArXiv.

[10]  Fred A Hamprecht,et al.  Multicut brings automated neurite segmentation closer to human performance , 2017, Nature Methods.

[11]  J. Ellenberg,et al.  Postmitotic nuclear pore assembly proceeds by radial dilation of small membrane openings , 2017, Nature Structural & Molecular Biology.

[12]  G. W. Knott,et al.  Neuron Segmentation With High-Level Biological Priors , 2018, IEEE Transactions on Medical Imaging.

[13]  Rangasami L. Kashyap,et al.  Building Skeleton Models via 3-D Medial Surface/Axis Thinning Algorithms , 1994, CVGIP Graph. Model. Image Process..

[14]  Ullrich Köthe,et al.  Ilastik: Interactive learning and segmentation toolkit , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[15]  Hanspeter Pfister,et al.  Efficient Correction for EM Connectomics with Skeletal Representation , 2018, BMVC.

[16]  Amos Fiat,et al.  Correlation clustering in general weighted graphs , 2006, Theor. Comput. Sci..

[17]  A. L. Eberle,et al.  High-resolution, high-throughput imaging with a multibeam scanning electron microscope , 2015, Journal of microscopy.

[18]  Stephan Saalfeld,et al.  Synaptic Cleft Segmentation in Non-Isotropic Volume Electron Microscopy of the Complete Drosophila Brain , 2018, MICCAI.

[19]  Fred A. Hamprecht,et al.  Automated Detection and Segmentation of Synaptic Contacts in Nearly Isotropic Serial Electron Microscopy Images , 2011, PloS one.

[20]  Peter Li,et al.  Solving Large Multicut Problems for Connectomics via Domain Decomposition , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[21]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[22]  Ullrich Köthe,et al.  The Mutex Watershed: Efficient, Parameter-Free Image Partitioning , 2018, ECCV.

[23]  Joachim M. Buhmann,et al.  Crowdsourcing the creation of image segmentation algorithms for connectomics , 2015, Front. Neuroanat..

[24]  Feng Li,et al.  The complete connectome of a learning and memory centre in an insect brain , 2017, Nature.

[25]  Christopher J. Peddie,et al.  3D correlative light and electron microscopy of cultured cells using serial blockface scanning electron microscopy , 2017, Journal of Cell Science.

[26]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[27]  Thomas Brox,et al.  Efficient Decomposition of Image and Mesh Graphs by Lifted Multicuts , 2015, ICCV.

[28]  Larry Lindsey,et al.  High-precision automated reconstruction of neurons with flood-filling networks , 2017, Nature Methods.

[29]  Christopher J. Obara,et al.  Increased spatiotemporal resolution reveals highly dynamic dense tubular matrices in the peripheral ER , 2016, Science.

[30]  Ullrich Köthe,et al.  An Efficient Fusion Move Algorithm for the Minimum Cost Lifted Multicut Problem , 2016, ECCV.

[31]  M. R. Rao,et al.  The partition problem , 1993, Math. Program..

[32]  S. Brenner,et al.  The structure of the nervous system of the nematode Caenorhabditis elegans. , 1986, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[33]  H. Sebastian Seung,et al.  An Error Detection and Correction Framework for Connectomics , 2017, NIPS.

[34]  Hayk Saribekyan,et al.  A Multi-Pass Approach to Large-Scale Connectomics , 2016, ArXiv.

[35]  Nir Shavit,et al.  Morphological Error Detection in 3D Segmentations , 2017, ArXiv.

[36]  Ullrich Köthe,et al.  Asymmetric Cuts: Joint Image Labeling and Partitioning , 2014, GCPR.

[37]  Xi Chen,et al.  An effective fully deep convolutional neural networks for mitochondria segmentation based on ATUM-SEM , 2018, Medical Imaging.

[38]  Bjoern Andres,et al.  Analysis and Optimization of Graph Decompositions by Lifted Multicuts , 2015, ICML.

[39]  Louis K. Scheffer,et al.  Synaptic circuits and their variations within different columns in the visual system of Drosophila , 2015, Proceedings of the National Academy of Sciences.

[40]  Jianbo Shi,et al.  Machine Learning of Hierarchical Clustering to Segment 2D and 3D Images , 2013, PloS one.

[41]  Marina Meila,et al.  Comparing Clusterings by the Variation of Information , 2003, COLT.