Random Walks on Graphs for Salient Object Detection in Images

We formulate the problem of salient object detection in images as an automatic labeling problem on the vertices of a weighted graph. The seed (labeled) nodes are first detected using Markov random walks performed on two different graphs that represent the image. While the global properties of the image are computed from the random walk on a complete graph, the local properties are computed from a sparse k-regular graph. The most salient node is selected as the one which is globally most isolated but falls on a locally compact object. A few background nodes and salient nodes are further identified based upon the random walk based hitting time to the most salient node. The salient nodes and the background nodes will constitute the labeled nodes. A new graph representation of the image that represents the saliency between nodes more accurately, the “pop-out graph” model, is computed further based upon the knowledge of the labeled salient and background nodes. A semisupervised learning technique is used to determine the labels of the unlabeled nodes by optimizing a smoothness objective label function on the newly created “pop-out graph” model along with some weighted soft constraints on the labeled nodes.

[1]  John N. Tsitsiklis,et al.  Introduction to Probability , 2002 .

[2]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[3]  Xing Xie,et al.  A visual attention model for adapting images on small displays , 2003, Multimedia Systems.

[4]  Simone Frintrop,et al.  Goal-Directed Search with a Top-Down Modulated Computational Attention System , 2005, DAGM-Symposium.

[5]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[6]  Fred Stentiford,et al.  Attention Based Auto Image Cropping , 2007, ICVS 2007.

[7]  David J. Aldous,et al.  Lower bounds for covering times for reversible Markov chains and random walks on graphs , 1989 .

[8]  Michael Brady,et al.  Saliency, Scale and Image Description , 2001, International Journal of Computer Vision.

[9]  Antonio Torralba,et al.  Top-down control of visual attention in object detection , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[10]  Jean Ponce,et al.  Segmentation by transduction , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Deepu Rajan,et al.  Random walks on graphs to model saliency in images , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  L. D. Costa Visual Saliency and Attention as Random Walks on Complex Networks , 2006, physics/0603025.

[13]  Nuno Vasconcelos,et al.  Discriminant Saliency for Visual Recognition from Cluttered Scenes , 2004, NIPS.

[14]  David Salesin,et al.  Gaze-based interaction for semi-automatic photo cropping , 2006, CHI.

[15]  Liming Zhang,et al.  Spatio-temporal Saliency detection using phase spectrum of quaternion fourier transform , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Nuno Vasconcelos,et al.  Bottom-up saliency is a discriminant process , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[17]  Christof Koch,et al.  Modeling attention to salient proto-objects , 2006, Neural Networks.

[18]  Patrick Le Callet,et al.  A coherent computational approach to model bottom-up visual attention , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Wei-Ying Ma,et al.  Data-driven approach for bridging the cognitive gap in image retrieval , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[20]  Alper Yilmaz,et al.  Interactive image segmentation using probabilistic hypergraphs , 2010, Pattern Recognit..

[21]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[22]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  John K. Tsotsos,et al.  Saliency Based on Information Maximization , 2005, NIPS.

[24]  Jang-Kyoo Shin,et al.  Biologically Inspired Saliency Map Model for Bottom-up Visual Attention , 2002, Biologically Motivated Computer Vision.

[25]  Deepu Rajan,et al.  Salient Region Detection by Modeling Distributions of Color and Orientation , 2009, IEEE Transactions on Multimedia.

[26]  Charles M. Grinstead,et al.  Introduction to probability , 1999, Statistics for the Behavioural Sciences.

[27]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.