Supervoxel-Based Segmentation of EM Image Stacks with Learned Shape Features

Immense amounts of high resolution data are now routinely produced thanks to recent advances in EM imaging. While a strong demand for automated analysis now exists, it is stifled by the lack of robust automatic 3D segmentation techniques. State-of-the-art Computer Vision algorithms designed to operate on natural 2D images tend to perform poorly when applied to EM image stacks for a number of reasons. The sheer size of a typical EM image stack renders many segmentation schemes intractable. Most approaches rely on local statistics that easily become confused when confronted with the noise and textures found within EM image stacks. The assumption that strong image gradients always correspond to object boundaries is violated by cluttered membranes belonging to numerous objects. In this work, we propose an automated graph partitioning scheme that addresses these issues. It reduces the computational complexity by operating on supervoxels instead of voxels, incorporates global shape features capable of describing the 3D shape of the target objects, and learns to recognize the distinctive appearance of true boundaries. Our experiments demonstrate that, when applied to segment mitochondria from neural tissue, our approach closely matches the performance of human annotators and outperforms a state-of-the-art 3D segmentation technique.

[1]  R. Achanta Finding Objects of Interest in Images using Saliency and Superpixels , 2011 .

[2]  Ross T. Whitaker,et al.  Detection of neuron membranes in electron microscopy images using a serial neural network architecture , 2010, Medical Image Anal..

[3]  Srinivas C. Turaga,et al.  Machines that learn to segment images: a crucial technology for connectomics , 2010, Current Opinion in Neurobiology.

[4]  Paria Mehrani,et al.  Superpixels and Supervoxels in an Energy Optimization Framework , 2010, ECCV.

[5]  Amelio Vázquez Reina,et al.  Radon-Like features and their application to connectomics , 2010, CVPR Workshops.

[6]  Joachim M. Buhmann,et al.  Neuron geometry extraction by perceptual grouping in ssTEM images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Cristian Sminchisescu,et al.  Constrained parametric min-cuts for automatic object segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Ronen Basri,et al.  Co-clustering of image segments using convex optimization applied to EM neuronal reconstruction , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Joost van de Weijer,et al.  Harmony potentials for joint classification and segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  H. Sebastian Seung,et al.  Boundary Learning by Optimization with Topological Constraints , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Vincent Lepetit,et al.  DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Joseph F. Murray,et al.  Convolutional Networks Can Learn to Generate Affinity Graphs for Image Segmentation , 2010, Neural Computation.

[13]  Sven J. Dickinson,et al.  TurboPixels: Fast Superpixels Using Geometric Flows , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Vincent Lepetit,et al.  Fast Ray features for learning irregular shapes , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[15]  Stefano Soatto,et al.  Class segmentation and object localization with superpixel neighborhoods , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[16]  Yuri Boykov,et al.  Globally optimal segmentation of multi-region objects , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  Tolga Tasdizen,et al.  Automatic markup of neural cell membranes using boosted decision stumps , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[18]  Amelio Vázquez Reina,et al.  Multiphase geometric couplings for the segmentation of neural processes , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Alexander G. Gray,et al.  Automatic joint classification and segmentation of whole cell 3D images , 2009, Pattern Recognit..

[20]  Yoonsuck Choe,et al.  3D volume extraction of densely packed cells in EM data stack by forward and backward graph cuts , 2009, 2009 IEEE Symposium on Computational Intelligence for Multimedia Signal and Vision Processing.

[21]  Jens Rittscher,et al.  Spatio-temporal cell cycle phase analysis using level sets and fast marching methods , 2009, Medical Image Anal..

[22]  Ross T. Whitaker,et al.  Axon tracking in serial block-face scanning electron microscopy , 2009, Medical Image Anal..

[23]  Yuriy Mishchenko,et al.  Automation of 3D reconstruction of neural tissue from large volume of conventional serial section transmission electron micrographs , 2009, Journal of Neuroscience Methods.

[24]  Filip Malmberg,et al.  Binarization of Phase Contrast Volume Images of Fibrous Materials - A Case Study , 2009, VISAPP.

[25]  B. S. Manjunath,et al.  Graph cut segmentation of neuronal structures from transmission electron micrographs , 2008, 2008 15th IEEE International Conference on Image Processing.

[26]  Stefano Soatto,et al.  Quick Shift and Kernel Methods for Mode Seeking , 2008, ECCV.

[27]  Ross T. Whitaker,et al.  An optimal-path approach for neural circuit reconstruction , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[28]  Qiang Ji,et al.  Shape-Driven Three-Dimensional Watersnake Segmentation of Biological Membranes in Electron Tomography , 2008, IEEE Transactions on Medical Imaging.

[29]  G. Knott,et al.  Serial Section Scanning Electron Microscopy of Adult Brain Tissue Using Focused Ion Beam Milling , 2008, The Journal of Neuroscience.

[30]  Anat Levin,et al.  Learning to Combine Bottom-Up and Top-Down Segmentation , 2006, International Journal of Computer Vision.

[31]  K. Harris,et al.  Mitochondria detection in electron microscopy images , 2008 .

[32]  Joseph F. Murray,et al.  Supervised Learning of Image Restoration with Convolutional Networks , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[33]  Aly A. Farag,et al.  Graph Cuts Framework for Kidney Segmentation with Prior Shape Constraints , 2007, MICCAI.

[34]  Vladimir Kolmogorov,et al.  Minimizing Nonsubmodular Functions with Graph Cuts-A Review , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Nikos Komodakis,et al.  Fast, Approximately Optimal Solutions for Single and Dynamic MRFs , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Andrew W. Fitzgibbon,et al.  Learning Class-Specific Edges for Object Detection and Segmentation , 2006, ICVGIP.

[37]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[38]  Luc Van Gool,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[39]  Ross T. Whitaker,et al.  Enhancement of cell boundaries in transmission electron microscopy images , 2005, IEEE International Conference on Image Processing 2005.

[40]  Vladimir Kolmogorov,et al.  What metrics can be approximated by geo-cuts, or global optimization of length/area and flux , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[41]  Andrew Zisserman,et al.  OBJ CUT , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[42]  Tao Zhang,et al.  Interactive graph cut based segmentation with shape priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[43]  Cordelia Schmid,et al.  IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2004, Washington, DC, USA, June 27 - July 2, 2004 , 2004, CVPR Workshops.

[44]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[45]  Vladimir Kolmogorov,et al.  "GrabCut": interactive foreground extraction using iterated graph cuts , 2004, ACM Trans. Graph..

[46]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  B. Williamson Biology: Exploring Life , 2004 .

[48]  R. Zabih,et al.  What energy functions can be minimized via graph cuts , 2004 .

[49]  William T. Freeman,et al.  Understanding belief propagation and its generalizations , 2003 .

[50]  Topi Mäenpää,et al.  The local binary pattern approach to texture analysis - extensions and applications , 2003 .

[51]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[52]  D. Mastronarde,et al.  Organellar relationships in the Golgi region of the pancreatic beta cell line, HIT-T15, visualized by high resolution electron tomography , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[53]  Guillermo Sapiro,et al.  Segmenting neurons in electronic microscopy via geometric tracing , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[54]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[55]  C. Mannella,et al.  Reconsidering mitochondrial structure: new views of an old organelle. , 1997, Trends in biochemical sciences.

[56]  Pascal Fua,et al.  Model-Based Optimization: Accurate and Consistent Site Modeling , 1996 .

[57]  D. Greig,et al.  Exact Maximum A Posteriori Estimation for Binary Images , 1989 .

[58]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .