Salient object detection based on super-pixel clustering and unified low-rank representation

In this paper, we present a novel salient object detection method, efficiently combining Laplacian sparse subspace clustering (LSSC) and unified low-rank representation (ULRR). Unlike traditional low-rank matrix recovery (LRMR) based saliency detection methods which mainly extract saliency from pixels or super-pixels, our method advocates the saliency detection on the super-pixel clusters generated by LSSC. By doing so, our method succeeds in extracting large-size salient objects from cluttered backgrounds, against the detection of small-size salient objects from simple backgrounds obtained by most existing work. The entire algorithm is carried out in two stages: region clustering and cluster saliency detection. In the first stage, the input image is segmented into many super-pixels, and on top of it, they are further grouped into different clusters by using LSSC. Each cluster contains multiple super-pixels having similar features (e.g., colors and intensities), and may correspond to a part of a salient object in the foreground or a local region in the background. In the second stage, we formulate the saliency detection of each super-pixel cluster as a unified low-rankness and sparsity pursuit problem using a ULRR model, which integrates a Laplacian regularization term with respect to the sparse error matrix into the traditional low-rank representation (LRR) model. The whole model is based on a sensible cluster-consistency assumption that the spatially adjacent super-pixels within the same cluster should have similar saliency values, similar representation coefficients as well as similar reconstruction errors. In addition, we construct a primitive dictionary for the ULRR model in terms of the local-global color contrast of each super-pixel. On top of it, a global saliency measure covering the representation coefficients and a local saliency measure considering the sparse reconstruction errors are jointly employed to define the final saliency measure. Comprehensive experiments over diverse publicly available benchmark data sets demonstrate the validity of the proposed method.

[1]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[2]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[3]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[4]  Shi-Min Hu,et al.  Sketch2Photo: internet image montage , 2009, ACM Trans. Graph..

[5]  Jian Sun,et al.  Geodesic Saliency Using Background Priors , 2012, ECCV.

[6]  Larry S. Davis,et al.  Learning Structured Low-Rank Representations for Image Classification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  V. Akhil.  SALIENT REGION DETECTION via HIGH-DIMENSIONAL COLOR TRANSFORM AND LOCAL SPATIAL SUPPORT , 2018 .

[8]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[9]  Ling Shao,et al.  Saliency-aware image-to-class distances for image classification , 2015, Neurocomputing.

[10]  Junwei Han,et al.  DHSNet: Deep Hierarchical Saliency Network for Salient Object Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Liming Zhang,et al.  A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression , 2010, IEEE Transactions on Image Processing.

[12]  Jie Yang,et al.  Saliency detection based on extended boundary prior with foci of attention , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Narendra Ahuja,et al.  Robust Visual Tracking Via Consistent Low-Rank Sparse Learning , 2014, International Journal of Computer Vision.

[14]  Ying Wu,et al.  A unified approach to salient object detection via low rank matrix recovery , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[16]  Nanning Zheng,et al.  Automatic salient object segmentation based on context and shape prior , 2011, BMVC.

[17]  Ioannis Rigas,et al.  Efficient modeling of visual saliency based on local sparse representation and the use of hamming distance , 2015, Comput. Vis. Image Underst..

[18]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Paul M. de Zeeuw,et al.  Fast saliency-aware multi-modality image fusion , 2013, Neurocomputing.

[20]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[21]  Junchi Yan,et al.  Visual Saliency Detection via Sparsity Pursuit , 2010, IEEE Signal Processing Letters.

[22]  Ling Shao,et al.  Cosaliency Detection Based on Intrasaliency Prior Transfer and Deep Intersaliency Mining , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Huchuan Lu,et al.  Salient object detection via bootstrap learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Gabriela Csurka,et al.  A framework for visual saliency detection with applications to image thumbnailing , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[25]  Martin D. Levine,et al.  Visual Saliency Based on Scale-Space Analysis in the Frequency Domain , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Christof Koch,et al.  Image Signature: Highlighting Sparse Salient Regions , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Zhixun Su,et al.  Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation , 2011, NIPS.

[29]  John K. Tsotsos,et al.  Saliency Based on Information Maximization , 2005, NIPS.

[30]  Pabitra Mitra,et al.  A dense subgraph based algorithm for compact salient image region detection , 2016, Comput. Vis. Image Underst..

[31]  Shuicheng Yan,et al.  Multi-task low-rank affinity pursuit for image segmentation , 2011, 2011 International Conference on Computer Vision.

[32]  Lihi Zelnik-Manor,et al.  What Makes a Patch Distinct? , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  James M. Rehg,et al.  The Secrets of Salient Object Segmentation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Lihi Zelnik-Manor,et al.  Context-Aware Saliency Detection , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Zengchang Qin,et al.  Multifocus image fusion based on robust principal component analysis , 2013, Pattern Recognit. Lett..

[36]  Huchuan Lu,et al.  Bayesian Saliency via Low and mid Level Cues , 2022 .

[37]  Guoying Zhao,et al.  Background Subtraction Based on Low-Rank and Structured Sparse Decomposition , 2015, IEEE Transactions on Image Processing.

[38]  Wei Liu,et al.  Saliency propagation from simple to difficult , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Sk Subhan,et al.  Dense and Sparse Reconstruction Error Based Saliency Descriptor , 2018 .

[40]  David A. Forsyth,et al.  Generalizing motion edits with Gaussian processes , 2009, ACM Trans. Graph..

[41]  Jian Sun,et al.  Saliency Optimization from Robust Background Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Isabelle Bloch,et al.  Sequential model-based segmentation and recognition of image structures driven by visual features and spatial relations , 2012, Comput. Vis. Image Underst..

[43]  Dewen Hu,et al.  Salient Region Detection via Integrating Diffusion-Based Compactness and Local Contrast , 2015, IEEE Transactions on Image Processing.

[44]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[46]  Haisheng Li,et al.  Saliency detection using two-stage scoring , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[47]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Changsheng Xu,et al.  Low-Rank Sparse Coding for Image Classification , 2013, 2013 IEEE International Conference on Computer Vision.

[49]  Liming Zhang,et al.  Spatio-temporal Saliency detection using phase spectrum of quaternion fourier transform , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Qiaosong Wang,et al.  GraB: Visual Saliency via Novel Graph Model and Background Priors , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  R. Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[52]  Shigang Wang,et al.  Local graph regularized sparse reconstruction for salient object detection , 2016, Neurocomputing.

[53]  Xin Liu,et al.  Background subtraction based on low-rank and structured sparse decomposition. , 2015, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[54]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Jian Yu,et al.  Saliency Detection by Multitask Sparsity Pursuit , 2012, IEEE Transactions on Image Processing.

[58]  Dattaguru V Kamat.  A framework for visual saliency detection with applications to image thumbnailing , 2009 .

[59]  Stephen J. Wright,et al.  Sparse reconstruction by separable approximation , 2009, IEEE Trans. Signal Process..

[60]  Li Xu,et al.  Hierarchical Image Saliency Detection on Extended CSSD , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[62]  HongJiang Zhang,et al.  Contrast-based image attention analysis by using fuzzy growing , 2003, MULTIMEDIA '03.