Spherical Superpixel Segmentation

These days, superpixel algorithms are widely used in computer vision and multimedia applications. However, existing algorithms are designed for planar images, which are less suited to deal with wide angle images. In this paper, we present a superpixel segmentation method for $\text{360}^\circ$ spherical images. Unlike previous methods, our approach explicitly considers the geometry for spherical images and makes clustering to spherical image pixels. It starts with the seeds defined by Hammersley points sampled on the sphere, then iterates between assignment step and update step, which are both based on the distance metric respecting spherical geometry. We evaluate our method on the transformed Berkeley segmentation dataset and panorama segmentation dataset collected by ourselves. Experimental results show that our method can gain better performance in terms of adherence to image boundaries and superpixel structural regularity. Furthermore, superpixels generated by our method can reserve the coherence across image boundaries and all have closed contours.

[1]  Chong Wang,et al.  Superpixel-Based Hand Gesture Recognition With Kinect Depth Camera , 2015, IEEE Transactions on Multimedia.

[2]  Rama Chellappa,et al.  Entropy rate superpixel segmentation , 2011, CVPR 2011.

[3]  Yue Gao,et al.  Representative Discovery of Structure Cues for Weakly-Supervised Image Segmentation , 2014, IEEE Transactions on Multimedia.

[4]  Wei Feng,et al.  SPHORB: A Fast and Robust Binary Feature on the Sphere , 2014, International Journal of Computer Vision.

[5]  Weisi Lin,et al.  A Universal Framework for Salient Object Detection , 2016, IEEE Transactions on Multimedia.

[6]  Samuel R. Buss,et al.  Spherical averages and applications to spherical splines and interpolation , 2001, TOGS.

[7]  Stefano Soatto,et al.  Class segmentation and object localization with superpixel neighborhoods , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[8]  Xiaochun Cao,et al.  Regularity Preserved Superpixels and Supervoxels , 2014, IEEE Transactions on Multimedia.

[9]  Jianjun Cui,et al.  Equidistribution on the Sphere , 1997, SIAM J. Sci. Comput..

[10]  Rainer Stiefelhagen,et al.  An evaluation of the compactness of superpixels , 2014, Pattern Recognit. Lett..

[11]  Hui Zhang,et al.  Efficient 3D Room Shape Recovery from a Single Panorama , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Takayuki Okatani,et al.  Change Detection from a Street Image Pair using CNN Features and Superpixel Segmentation , 2015, BMVC.

[13]  手塚 集 Uniform random numbers : theory and practice , 1995 .

[14]  Sven J. Dickinson,et al.  TurboPixels: Fast Superpixels Using Geometric Flows , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Yinda Zhang,et al.  PanoContext: A Whole-Room 3D Context Model for Panoramic Scene Understanding , 2014, ECCV.

[16]  Shuicheng Yan,et al.  Fashion Parsing With Weak Color-Category Labels , 2014, IEEE Transactions on Multimedia.

[17]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[18]  Krista A. Ehinger,et al.  Recognizing scene viewpoint using panoramic place representation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Antonio Criminisi,et al.  TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation , 2006, ECCV.

[20]  Jana Kosecka,et al.  Detecting Changes in Images of Street Scenes , 2012, ACCV.

[21]  Hujun Bao,et al.  Spatio-Temporal Video Segmentation of Static Scenes and Its Applications , 2015, IEEE Transactions on Multimedia.

[22]  R. Osserman The isoperimetric inequality , 1978 .

[23]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Svetlana Lazebnik,et al.  Superparsing - Scalable Nonparametric Image Parsing with Superpixels , 2010, International Journal of Computer Vision.

[25]  Tien-Tsin Wong,et al.  Cube2Video: Navigate Between Cubic Panoramas in Real-Time , 2013, IEEE Transactions on Multimedia.

[26]  Ricardo Cabral,et al.  Piecewise Planar and Compact Floorplan Reconstruction from Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Jitendra Malik,et al.  Learning a classification model for segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[28]  Gui-Song Xia,et al.  A Computational Model for Object-Based Visual Saliency: Spreading Attention Along Gestalt Cues , 2016, IEEE Transactions on Multimedia.

[29]  Jana Kosecka,et al.  Piecewise planar city 3D modeling from street view panoramic sequences , 2009, CVPR.

[30]  Eric L. Miller,et al.  Multiple Hypothesis Video Segmentation from Superpixel Flows , 2010, ECCV.

[31]  George Drettakis,et al.  Depth synthesis and local warps for plausible image-based navigation , 2013, TOGS.

[32]  Paria Mehrani,et al.  Superpixels and Supervoxels in an Energy Optimization Framework , 2010, ECCV.

[33]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[34]  Luc Van Gool,et al.  SEEDS: Superpixels Extracted Via Energy-Driven Sampling , 2012, International Journal of Computer Vision.

[35]  Daniel Thalmann,et al.  Parsing the Hand in Depth Images , 2014, IEEE Transactions on Multimedia.

[36]  Liang Li,et al.  Maximum Cohesive Grid of Superpixels for Fast Object Localization , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Hongbin Zha,et al.  Structure-Sensitive Superpixels via Geodesic Distance , 2011, 2011 International Conference on Computer Vision.

[38]  Sing Bing Kang,et al.  Stereo for Image-Based Rendering using Image Over-Segmentation , 2007, International Journal of Computer Vision.

[39]  Sebastian Nowozin,et al.  On Parameter Learning in CRF-Based Approaches to Object Class Image Segmentation , 2010, ECCV.

[40]  Jiawan Zhang,et al.  Spherical superpixel segmentation , 2016, 2016 IEEE International Conference on Multimedia and Expo (ICME).

[41]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Denis Zorin,et al.  Correction of geometric perceptual distortions in pictures , 1995, SIGGRAPH.

[43]  Tien-Tsin Wong,et al.  Sampling with Hammersley and Halton Points , 1997, J. Graphics, GPU, & Game Tools.

[44]  Kurt Hornik,et al.  Spherical k-Means Clustering , 2012 .