Saliency Cuts on RGB-D Images

Saliency cuts aims to segment salient objects from a given saliency map. The existing saliency cuts methods focus on dealing with RGB images and videos, but ignore the exploration of depth cue, which limit their performance on RGB-D images. In this paper, we propose a novel saliency cuts method on RGB-D images, which utilizes both color and depth cues to segment salient objects. Given a saliency map, we first generate segmentation seeds with adaptive triple thresholding. Next, we extend GrabCut by combining depth cue, and use it to generate a roughly labeled map. Finally, we refine the boundary of the salient object adaptively, and produce an accurate binary mask. To the best of our knowledge, this method is the first specific saliency cuts method for RGB-D images. We validated the proposed method on the largest RGB-D image dataset for salient object detection, named NJU2000. The experimental results demonstrate that our method outperforms the state-of-the-art methods.

[1]  Meng Wang,et al.  Learning Visual Semantic Relationships for Efficient Visual Retrieval , 2015, IEEE Transactions on Big Data.

[2]  Meng Wang,et al.  Oracle in Image Search: A Content-Based Approach to Performance Prediction , 2012, TOIS.

[3]  Yang Wang,et al.  Salient Object Segmentation via Effective Integration of Saliency and Objectness , 2017, IEEE Transactions on Multimedia.

[4]  Jinhui Tang,et al.  Weakly Supervised Deep Matrix Factorization for Social Image Understanding , 2017, IEEE Transactions on Image Processing.

[5]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[6]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Dacheng Tao,et al.  Multi-View Object Retrieval via Multi-Scale Topic Models. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[9]  Ran Ju,et al.  Interactive RGB-D Image Segmentation Using Hierarchical Graph Cut and Geodesic Distance , 2015, PCM.

[10]  Hanqing Lu,et al.  Saliency Cuts: An automatic approach to object segmentation , 2008, 2008 19th International Conference on Pattern Recognition.

[11]  Yang Liu,et al.  Depth-aware salient object detection using anisotropic center-surround difference , 2015, Signal Process. Image Commun..

[12]  Ramesh C. Jain,et al.  Image annotation by kNN-sparse graph-based label propagation over noisily tagged web images , 2011, TIST.

[13]  Yiannis Kompatsiaris,et al.  Integrating Image Segmentation and Classification for Fuzzy Knowledge-Based Multimedia Indexing , 2009, MMM.

[14]  Meng Wang,et al.  Multimedia answering: enriching text QA with media information , 2011, SIGIR.

[15]  Cristian Sminchisescu,et al.  Video Object Segmentation by Salient Segment Chain Composition , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[16]  Patrick Pérez,et al.  Interactive Image Segmentation Using an Adaptive GMMRF Model , 2004, ECCV.

[17]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[18]  Changsheng Xu,et al.  User-Aware Image Tag Refinement via Ternary Semantic Analysis , 2012, IEEE Transactions on Multimedia.

[19]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[20]  Nick Barnes,et al.  Local Background Enclosure for RGB-D Salient Object Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[22]  Ning Xu,et al.  Object segmentation using graph cuts based active contours , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[23]  Ran Ju,et al.  Saliency cuts based on adaptive triple thresholding , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[24]  Huan Du,et al.  Depth-Aware Salient Object Detection and Segmentation via Multiscale Discriminative Saliency Fusion and Bootstrap Learning , 2017, IEEE Transactions on Image Processing.

[25]  Lei Huang,et al.  Saliency detection on sampled images for tag ranking , 2017, Multimedia Systems.

[26]  Meng Wang,et al.  Harvesting visual concepts for image search with complex queries , 2012, ACM Multimedia.

[27]  Roger Zimmermann,et al.  Flickr Circles: Aesthetic Tendency Discovery by Multi-View Regularized Topic Modeling , 2016, IEEE Transactions on Multimedia.

[28]  Jing Liu,et al.  Object proposal on RGB-D images via elastic edge boxes , 2017, Neurocomputing.

[29]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.