Bayesian Adaptive Superpixel Segmentation

Superpixels provide a useful intermediate image representation. Existing superpixel methods, however, suffer from at least some of the following drawbacks: 1) topology is handled heuristically; 2) the number of superpixels is either predefined or estimated at a prohibitive cost; 3) lack of adaptiveness. As a remedy, we propose a novel probabilistic model, self-coined Bayesian Adaptive Superpixel Segmentation (BASS), together with an efficient inference. BASS is a Bayesian nonparametric mixture model that also respects topology and favors spatial coherence. The optimizationbased and topology-aware inference is parallelizable and implemented in GPU. Quantitatively, BASS achieves results that are either better than the state-of-the-art or close to it, depending on the performance index and/or dataset. Qualitatively, we argue it achieves the best results; we demonstrate this by not only subjective visual inspection but also objective quantitative performance evaluation of the downstream application of face detection. Our code is available at https://github.com/uzielroy/BASS.

[1]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[2]  N. Deo,et al.  Graph-theoretic algorithms for image segmentation , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[3]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[4]  Ian Reid,et al.  gSLIC: a real-time implementation of SLIC superpixel segmentation , 2011 .

[5]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[6]  Bernt Schiele,et al.  What Makes for Effective Detection Proposals? , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[8]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Luc Van Gool,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[10]  John W. Fisher,et al.  A Video Representation Using Temporal Superpixels , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Soumya Ghosh,et al.  Nonparametric learning for layered segmentation of natural images , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Bastian Leibe,et al.  Superpixels: An evaluation of the state-of-the-art , 2016, Comput. Vis. Image Underst..

[13]  Shih-Fu Chang,et al.  Segmentation using superpixels: A bipartite graph partitioning approach , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Ronald M. Summers,et al.  A Bottom-Up Approach for Pancreas Segmentation Using Cascaded Superpixels and (Deep) Image Patch Labeling , 2015, IEEE Transactions on Image Processing.

[15]  Erik B. Sudderth.  Graphical models for visual object recognition and tracking , 2006 .

[16]  Andrew P. French,et al.  SMURFS: Superpixels from Multi-scale Refinement of Super-regions , 2016, BMVC.

[17]  Or Dinari,et al.  Distributed MCMC Inference in Dirichlet Process Mixture Models Using Julia , 2019, 2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID).

[18]  Peter V. Gehler,et al.  Superpixel Convolutional Networks Using Bilateral Inceptions , 2015, ECCV.

[19]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Sven J. Dickinson,et al.  TurboPixels: Fast Superpixels Using Geometric Flows , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Gregory Shakhnarovich,et al.  Image Segmentation by Cascaded Region Agglomeration , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Luc Van Gool,et al.  SEEDS: Superpixels Extracted via Energy-Driven Sampling , 2012, ECCV.

[23]  Li Li,et al.  Edge-based split-and-merge superpixel segmentation , 2015, 2015 IEEE International Conference on Information and Automation.

[24]  Stephen Gould,et al.  Decomposing a scene into geometric and semantically consistent regions , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[25]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[26]  Cordelia Schmid,et al.  Object Recognition by Integrating Multiple Image Segmentations , 2008, ECCV.

[27]  Sanja Fidler,et al.  Real-time coarse-to-fine topologically preserving segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Ronen Basri,et al.  Image Segmentation by Probabilistic Bottom-Up Aggregation and Cue Integration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Xiao Ma,et al.  Superpixel segmentation: A benchmark , 2017, Signal Process. Image Commun..

[30]  Jianbing Shen,et al.  Real-Time Superpixel Segmentation by DBSCAN Clustering Algorithm. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[31]  Nathan D. Cahill,et al.  SLIC superpixels for efficient graph-based dimensionality reduction of hyperspectral imagery , 2015, Defense + Security Symposium.

[32]  Richard S. Zemel,et al.  Learning and Incorporating Top-Down Cues in Image Segmentation , 2006, ECCV.

[33]  Stefano Soatto,et al.  Class segmentation and object localization with superpixel neighborhoods , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[34]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[36]  Jitendra Malik,et al.  Blobworld: A System for Region-Based Image Indexing and Retrieval , 1999, VISUAL.

[37]  Frank Nielsen,et al.  Earth Mover Distance on superpixels , 2010, 2010 IEEE International Conference on Image Processing.

[38]  Shai Avidan,et al.  Locally Orderless Tracking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Florent Lafarge,et al.  Image partitioning into convex polygons , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Alexei A. Efros,et al.  Geometric context from a single image , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[41]  Gerhard Winkler,et al.  Image Analysis, Random Fields and Markov Chain Monte Carlo Methods: A Mathematical Introduction , 2002 .

[42]  Paria Mehrani,et al.  Superpixels and Supervoxels in an Energy Optimization Framework , 2010, ECCV.

[43]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Hao Wu,et al.  Superpixels for Spatially Reinforced Bayesian Classification of Hyperspectral Images , 2015, IEEE Geoscience and Remote Sensing Letters.

[45]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[46]  John W. Fisher,et al.  A fast method for inferring high-quality simply-connected superpixels , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[47]  John W. Fisher,et al.  Parallel Sampling of DP Mixture Models using Sub-Cluster Splits , 2013, NIPS.

[48]  Jitendra Malik,et al.  Learning a classification model for segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[49]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[50]  Svetlana Lazebnik,et al.  Superparsing - Scalable Nonparametric Image Parsing with Superpixels , 2010, International Journal of Computer Vision.

[51]  Shuo Yang,et al.  WIDER FACE: A Face Detection Benchmark , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Alexei A. Efros,et al.  Recovering Occlusion Boundaries from an Image , 2011, International Journal of Computer Vision.