Computing object-based saliency via locality-constrained linear coding and conditional random fields

Predicting object location using a top-down saliency model has grown increasingly popular in recent years. In this work, we combine locality-constrained linear coding (LLC) with a conditional random field (CRF), and construct a top-down saliency model to generate a specific object-based saliency map. During the training phase, we use the LLC codes as the latent variables of the CRF model, and meanwhile learn a class-specific codebook by CRF modulation. In the testing phase, we use this top-down model to distinguish specific objects from a cluttered background. Finally, we evaluate the experimental results on the MSRA-B, Garz-02, Weizmann Horse, and Plane datasets by applying the developed object-based saliency model. The performance shows that our approach can not only improve the precision but also dramatically reduce the computational complexity.

[1]  Nuno Vasconcelos,et al.  Discriminant Saliency, the Detection of Suspicious Coincidences, and Applications to Visual Recognition , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Antonio Criminisi,et al.  TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context , 2007, International Journal of Computer Vision.

[3]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Tianli Yu,et al.  Kernelized structural SVM learning for supervised object segmentation , 2011, CVPR 2011.

[5]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[6]  Lianfa Bai,et al.  Saliency detection via Boolean and foreground in a dynamic Bayesian framework , 2017, The Visual Computer.

[7]  Matthew H Tong,et al.  SUN: Top-down saliency using natural statistics , 2009, Visual cognition.

[8]  P. Perona,et al.  Objects predict fixations better than early saliency. , 2008, Journal of vision.

[9]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[10]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[11]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Fahad Shahbaz Khan,et al.  Top-down color attention for object recognition , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[13]  K. Fujii,et al.  Visualization for the analysis of fluid motion , 2005, J. Vis..

[14]  Min Xu,et al.  A novel optimization framework for salient object detection , 2014, The Visual Computer.

[15]  Jean Ponce,et al.  Task-Driven Dictionary Learning , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Aykut Erdem,et al.  Top down saliency estimation via superpixel-based discriminative dictionaries , 2014, BMVC.

[17]  Peter Auer,et al.  Generic object recognition with boosting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Thomas S. Huang,et al.  Supervised translation-invariant sparse coding , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Min Xu,et al.  Saliency detection with color contrast based on boundary information and neighbors , 2014, The Visual Computer.

[20]  Sibel Tari,et al.  Top-down visual search in Wimmelbild , 2013, Electronic Imaging.

[21]  H. Teuber.  Physiological psychology. , 1955, Annual review of psychology.

[22]  Shimon Ullman,et al.  Combining Top-Down and Bottom-Up Segmentation , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[23]  Jian Sun,et al.  Saliency Optimization from Robust Background Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Yihong Gong,et al.  Locality-constrained Linear Coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Yihong Gong,et al.  Nonlinear Learning using Local Coordinate Coding , 2009, NIPS.

[26]  Jianzhong Wang,et al.  Multiview saliency detection based on improved multimanifold ranking , 2014, J. Electronic Imaging.

[27]  Weiren Shi,et al.  Region saliency detection via multi-feature on absorbing Markov chain , 2015, The Visual Computer.

[28]  Ramón López de Mántaras,et al.  Fast and robust object segmentation with the Integral Linear Classifier , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  Junjie Cao,et al.  A generalized nonlocal mean framework with object-level cues for saliency detection , 2015, The Visual Computer.

[30]  Thorsten Joachims,et al.  Cutting-plane training of structural SVMs , 2009, Machine Learning.

[31]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[32]  J. Wolfe,et al.  What attributes guide the deployment of visual attention and how do they do it? , 2004, Nature Reviews Neuroscience.

[33]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[34]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[35]  C. Kennard,et al.  The role of visual salience in directing eye movements in visual object agnosia , 2009, Current Biology.

[36]  Laurent Itti,et al.  Interesting objects are visually salient. , 2008, Journal of vision.

[37]  Ming-Hsuan Yang,et al.  Top-down visual saliency via joint CRF and dictionary learning , 2012, CVPR.

[38]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[39]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[40]  Radomír Mech,et al.  Minimum Barrier Salient Object Detection at 80 FPS , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[41]  Greg Mori,et al.  Max-margin hidden conditional random fields for human action recognition , 2009, CVPR.

[42]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, CVPR.

[43]  Guillermo Sapiro,et al.  Supervised Dictionary Learning , 2008, NIPS.

[44]  Trevor Darrell,et al.  Hidden Conditional Random Fields , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.