Intrinsic Manifold SLIC: A Simple and Efficient Method for Computing Content-Sensitive Superpixels

Superpixels are perceptually meaningful atomic regions that can effectively capture image features. Among various methods for computing uniform superpixels, simple linear iterative clustering (SLIC) is popular due to its simplicity and high performance. In this paper, we extend SLIC to compute content-sensitive superpixels, i.e., small superpixels in content-dense regions with high intensity or colour variation and large superpixels in content-sparse regions. Rather than using the conventional SLIC method that clusters pixels in <inline-formula><tex-math notation="LaTeX"> $\mathbb {R}^5$</tex-math><alternatives><inline-graphic xlink:href="liu-ieq1-2686857.gif"/></alternatives> </inline-formula>, we map the input image <inline-formula><tex-math notation="LaTeX">$I$</tex-math><alternatives> <inline-graphic xlink:href="liu-ieq2-2686857.gif"/></alternatives></inline-formula> to a 2-dimensional manifold <inline-formula><tex-math notation="LaTeX">$\mathcal {M}\subset \mathbb {R}^5$</tex-math><alternatives> <inline-graphic xlink:href="liu-ieq3-2686857.gif"/></alternatives></inline-formula>, whose area elements are a good measure of the content density in <inline-formula><tex-math notation="LaTeX">$I$</tex-math><alternatives> <inline-graphic xlink:href="liu-ieq4-2686857.gif"/></alternatives></inline-formula>. We propose a simple method, called <italic>intrinsic manifold SLIC</italic> (IMSLIC), for computing a geodesic centroidal Voronoi tessellation (GCVT)—a uniform tessellation—on <inline-formula><tex-math notation="LaTeX">$\mathcal {M}$</tex-math> <alternatives><inline-graphic xlink:href="liu-ieq5-2686857.gif"/></alternatives></inline-formula>, which induces the content-sensitive superpixels in <inline-formula><tex-math notation="LaTeX">$I$</tex-math><alternatives> <inline-graphic xlink:href="liu-ieq6-2686857.gif"/></alternatives></inline-formula>. In contrast to the existing algorithms, IMSLIC characterizes the content sensitivity by measuring areas of Voronoi cells on <inline-formula> <tex-math notation="LaTeX">$\mathcal {M}$</tex-math><alternatives><inline-graphic xlink:href="liu-ieq7-2686857.gif"/> </alternatives></inline-formula>. Using a simple and fast approximation to a closed-form solution, the method can compute the GCVT at a very low cost and guarantees that all Voronoi cells are simply connected. We thoroughly evaluate IMSLIC and compare it with eleven representative methods on the BSDS500 dataset and seven representative methods on the NYUV2 dataset. Computational results show that IMSLIC outperforms existing methods in terms of commonly used quality measures pertaining to superpixels such as compactness, adherence to boundaries, and achievable segmentation accuracy. We also evaluate IMSLIC and seven representative methods in an image contour closure application, and the results on two datasets, WHD and WSD, show that IMSLIC achieves the best foreground segmentation performance.

[1]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[2]  Joshua B. Tenenbaum,et al.  Global Versus Local Methods in Nonlinear Dimensionality Reduction , 2002, NIPS.

[3]  Rainer Stiefelhagen,et al.  An evaluation of the compactness of superpixels , 2014, Pattern Recognit. Lett..

[4]  Michael Beetz,et al.  Depth-adaptive superpixels , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[5]  Dimitri P. Bertsekas,et al.  Network optimization : continuous and discrete models , 1998 .

[6]  Sven J. Dickinson,et al.  Optimal Contour Closure by Superpixel Grouping , 2010, ECCV.

[7]  Sven J. Dickinson,et al.  TurboPixels: Fast Superpixels Using Geometric Flows , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Sven J. Dickinson,et al.  Optimal Image and Video Closure by Superpixel Grouping , 2012, International Journal of Computer Vision.

[9]  Kai Tang,et al.  Construction of Iso-Contours, Bisectors, and Voronoi Diagrams on Triangulated Surfaces , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Luc Van Gool,et al.  SEEDS: Superpixels Extracted Via Energy-Driven Sampling , 2012, International Journal of Computer Vision.

[11]  Dimitri P. Bertsekas,et al.  A simple and fast label correcting algorithm for shortest paths , 1993, Networks.

[12]  Rama Chellappa,et al.  Entropy rate superpixel segmentation , 2011, CVPR 2011.

[13]  Paria Mehrani,et al.  Superpixels and Supervoxels in an Energy Optimization Framework , 2010, ECCV.

[14]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[15]  Stefano Soatto,et al.  Class segmentation and object localization with superpixel neighborhoods , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[16]  D. Bertsekas,et al.  Parallel asynchronous label-correcting methods for shortest paths , 1996 .

[17]  Hongbin Zha,et al.  Structure-Sensitive Superpixels via Geodesic Distance , 2011, 2011 International Conference on Computer Vision.

[18]  Yong-Jin Liu.  Semi-Continuity of Skeletons in Two-Manifold and Discrete Voronoi Approximation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Horst Reiner,et al.  Introduction to Global Optimization. Second Edition , 2000 .

[20]  Jana Kosecka,et al.  Multi-view Superpixel Stereo in Urban Environments , 2010, International Journal of Computer Vision.

[21]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[23]  Ronen Basri,et al.  Image Segmentation by Probabilistic Bottom-Up Aggregation and Cue Integration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Leo Liberti,et al.  Introduction to Global Optimization , 2006 .

[25]  Jie Wang,et al.  VCells: Simple and Efficient Superpixels Using Edge-Weighted Centroidal Voronoi Tessellations , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Yong-Jin Liu,et al.  Manifold SLIC: A Fast Method to Compute Content-Sensitive Superpixels , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Shimon Ullman,et al.  Class-Specific, Top-Down Segmentation , 2002, ECCV.

[28]  Huchuan Lu,et al.  Superpixel tracking , 2011, 2011 International Conference on Computer Vision.

[29]  Umar Mohammed,et al.  Superpixel lattices , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Qiang Du,et al.  Constrained Centroidal Voronoi Tessellations for Surfaces , 2002, SIAM J. Sci. Comput..

[31]  Qiang Du,et al.  Centroidal Voronoi Tessellations: Applications and Algorithms , 1999, SIAM Rev..

[32]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  智一 吉田,et al.  Efficient Graph-Based Image Segmentationを用いた圃場図自動作成手法の検討 , 2014 .

[34]  Jie Wang,et al.  Image Segmentation Using Local Variation and Edge-Weighted Centroidal Voronoi Tessellations , 2011, IEEE Transactions on Image Processing.

[35]  Xiaohu Guo,et al.  Anisotropic Superpixel Generation Based on Mahalanobis Distance , 2016, Comput. Graph. Forum.