Boundary Learning by Optimization with Topological Constraints

Recent studies have shown that machine learning can improve the accuracy of detecting object boundaries in images. In the standard approach, a boundary detector is trained by minimizing its pixel-level disagreement with human boundary tracings. This naive metric is problematic because it is overly sensitive to boundary locations. This problem is solved by metrics provided with the Berkeley Segmentation Dataset, but these can be insensitive to topo-logical differences, such as gaps in boundaries. Furthermore, the Berkeley metrics have not been useful as cost functions for supervised learning. Using concepts from digital topology, we propose a new metric called the warping error that tolerates disagreements over boundary location, penalizes topological disagreements, and can be used directly as a cost function for learning boundary detection, in a method that we call Boundary Learning by Optimization with Topological Constraints (BLOTC). We trained boundary detectors on electron microscopic images of neurons, using both BLOTC and standard training. BLOTC produced substantially better performance on a 1.2 million pixel test set, as measured by both the warping error and the Rand index evaluated on segmentations generated from the boundary labelings. We also find our approach yields significantly better segmentation performance than either gPb-OWT-UCM or multiscale normalized cut, as well as Boosted Edge Learning trained directly on our data.

[1]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[2]  Azriel Rosenfeld,et al.  Digital topology: Introduction and survey , 1989, Comput. Vis. Graph. Image Process..

[3]  Gilles Bertrand,et al.  A new characterization of three-dimensional simple points , 1994, Pattern Recognition Letters.

[4]  Gilles Bertrand,et al.  A Boolean characterization of three-dimensional simple points , 1996, Pattern Recognition Letters.

[5]  Chris Pudney,et al.  Distance-Ordered Homotopic Thinning: A Skeletonization Algorithm for 3D Digital Images , 1998, Comput. Vis. Image Underst..

[6]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[7]  Azriel Rosenfeld,et al.  Topology-Preserving Deformations of Two-Valued Digital Pictures , 1998, Graph. Model. Image Process..

[8]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .

[9]  木本 浩司 Microscopy and Microanalysis 2002 , 2002, Microscopy and Microanalysis.

[10]  Jitendra Malik,et al.  Learning to Detect Natural Image Boundaries Using Brightness and Texture , 2002, NIPS.

[11]  Jitendra Malik,et al.  Learning affinity functions for image segmentation: combining patch-based and gradient-based approaches , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[12]  Xiao Han,et al.  A Topology Preserving Level Set Method for Geometric Deformable Models , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  W. Denk,et al.  Serial Block-Face Scanning Electron Microscopy to Reconstruct Three-Dimensional Tissue Nanostructure , 2004, PLoS biology.

[15]  Jianbo Shi,et al.  Spectral segmentation with multiscale graph decomposition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[16]  N. Kasthuri,et al.  Automating the Collection of Ultrathin Serial Sections for Large Volume TEM Reconstructions , 2006, Microscopy and Microanalysis.

[17]  Zhuowen Tu,et al.  Supervised Learning of Edges and Object Boundaries , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[18]  Martial Hebert,et al.  Toward Objective Evaluation of Image Segmentation Algorithms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Joseph F. Murray,et al.  Supervised Learning of Image Restoration with Convolutional Networks , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[20]  Ullrich Köthe,et al.  Segmentation of SBFSEM Volume Data of Neural Tissue by Hierarchical Classification , 2008, DAGM-Symposium.

[21]  Jitendra Malik,et al.  Using contours to detect and localize junctions in natural images , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Dimitris Samaras,et al.  Topology cuts: A novel min-cut/max-flow algorithm for topology preserving segmentation in N-D images , 2008, Comput. Vis. Image Underst..

[23]  Vladimir Kolmogorov,et al.  Graph cut based image segmentation with connectivity priors , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Kevin L. Briggman,et al.  3D structural imaging of the brain with photons and electrons , 2008, Current Opinion in Neurobiology.

[25]  J. Sanes,et al.  Ome sweet ome: what can the genome tell us about the connectome? , 2008, Current Opinion in Neurobiology.

[26]  H. Sebastian Seung,et al.  Maximin affinity learning of image segmentation , 2009, NIPS.

[27]  Ross T. Whitaker,et al.  Axon tracking in serial block-face scanning electron microscopy , 2009, Medical Image Anal..

[28]  Sebastian Nowozin,et al.  Global connectivity potentials for random field models , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Jitendra Malik,et al.  From contours to regions: An empirical evaluation , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Amelio Vázquez Reina,et al.  Multiphase geometric couplings for the segmentation of neural processes , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  H. Sebastian Seung,et al.  Reading the Book of Memory: Sparse Sampling versus Dense Mapping of Connectomes , 2009, Neuron.

[32]  Viren Jain,et al.  Machine learning of image analysis with convolutional networks and topological constraints , 2010 .

[33]  Joseph F. Murray,et al.  Convolutional Networks Can Learn to Generate Affinity Graphs for Image Segmentation , 2010, Neural Computation.