Spatiochromatic Context Modeling for Color Saliency Analysis

Visual saliency is one of the most noteworthy perceptual abilities of human vision. Recent progress in cognitive psychology suggests that: 1) visual saliency analysis is mainly completed by the bottom-up mechanism consisting of feedforward low-level processing in primary visual cortex (area V1) and 2) color interacts with spatial cues and is influenced by the neighborhood context, and thus it plays an important role in a visual saliency analysis. From a computational perspective, the most existing saliency modeling approaches exploit multiple independent visual cues, irrespective of their interactions (or are not computed explicitly), and ignore contextual influences induced by neighboring colors. In addition, the use of color is often underestimated in the visual saliency analysis. In this paper, we propose a simple yet effective color saliency model that considers color as the only visual cue and mimics the color processing in V1. Our approach uses region-/boundary-defined color features with spatiochromatic filtering by considering local color-orientation interactions, therefore captures homogeneous color elements, subtle textures within the object and the overall salient object from the color image. To account for color contextual influences, we present a divisive normalization method for chromatic stimuli through the pooling of contrary/complementary color units. We further define a color perceptual metric over the entire scene to produce saliency maps for color regions and color boundaries individually. These maps are finally globally integrated into a one single saliency map. The final saliency map is produced by Gaussian blurring for robustness. We evaluate the proposed method on both synthetic stimuli and several benchmark saliency data sets from the visual saliency analysis to salient object detection. The experimental results demonstrate that the use of color as a unique visual cue achieves competitive results on par with or better than 12 state-of-the-art approaches.

[1]  Min-Chun Hu,et al.  Learning and Recognition of On-Premise Signs From Weakly Labeled Street View Images , 2014, IEEE Transactions on Image Processing.

[2]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[3]  R. Freeman,et al.  Origins of cross-orientation suppression in the visual cortex. , 2006, Journal of neurophysiology.

[4]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Xiaochun Cao,et al.  Video color conceptualization using optimization , 2014, Science China Information Sciences.

[6]  Hanqing Lu,et al.  Saliency Cuts: An automatic approach to object segmentation , 2008, 2008 19th International Conference on Pattern Recognition.

[7]  D. Heeger Normalization of cell responses in cat striate cortex , 1992, Visual Neuroscience.

[8]  Christof Koch,et al.  Learning a saliency map using fixated locations in natural scenes. , 2011, Journal of vision.

[9]  Li Zhaoping,et al.  Properties of V1 Neurons Tuned to Conjunctions of Visual Features: Application of the V1 Saliency Hypothesis to Visual Search behavior , 2012, PloS one.

[10]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Jian Sun,et al.  Geodesic Saliency Using Background Priors , 2012, ECCV.

[12]  Xuelong Li,et al.  Spatial-Aware Object-Level Saliency Prediction by Learning Graphlet Hierarchies , 2015, IEEE Transactions on Industrial Electronics.

[13]  Huchuan Lu,et al.  Saliency Detection via Dense and Sparse Reconstruction , 2013, 2013 IEEE International Conference on Computer Vision.

[14]  Qi Zhao,et al.  Learning saliency-based visual attention: A review , 2013, Signal Process..

[15]  Qi Zhao,et al.  Learning to predict eye fixations for semantic contents using multi-layer sparse network , 2014, Neurocomputing.

[16]  Lei Guo,et al.  An Object-Oriented Visual Saliency Detection Framework Based on Sparse Coding Representations , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  James L. Crowley,et al.  Local Scale Selection for Gaussian Based Description Techniques , 2000, ECCV.

[18]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[20]  Lihi Zelnik-Manor,et al.  Context-aware saliency detection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Aykut Erdem,et al.  Visual saliency estimation by nonlinearly integrating features using region covariances. , 2013, Journal of vision.

[22]  Nuno Vasconcelos,et al.  Object recognition with hierarchical discriminant saliency networks , 2014, Front. Comput. Neurosci..

[23]  M. Carandini,et al.  Normalization as a canonical neural computation , 2011, Nature Reviews Neuroscience.

[24]  Feng Wu,et al.  Background Prior-Based Salient Object Detection via Deep Reconstruction Residual , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Stan Sclaroff,et al.  Saliency Detection: A Boolean Map Approach , 2013, 2013 IEEE International Conference on Computer Vision.

[26]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[27]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[28]  Bevil R. Conway,et al.  Spatial and Temporal Properties of Cone Signals in Alert Macaque Primary Visual Cortex , 2006, The Journal of Neuroscience.

[29]  Brad Wyble,et al.  Detecting meaning in RSVP at 13 ms per picture , 2013, Attention, perception & psychophysics.

[30]  Li Yang,et al.  Exploring feature sets for two-phase biomedical named entity recognition using semi-CRFs , 2013, Knowledge and Information Systems.

[31]  Yi Yang,et al.  Effective transfer tagging from image to video , 2013, TOMCCAP.

[32]  Xuelong Li,et al.  Two-Stage Learning to Predict Human Eye Fixations via SDAEs , 2016, IEEE Transactions on Cybernetics.

[33]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[34]  Jian Yang,et al.  Sparse Tensor Discriminant Color Space for Face Verification , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Cordelia Schmid,et al.  Discriminative spatial saliency for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Nuno Vasconcelos,et al.  The discriminant center-surround hypothesis for bottom-up saliency , 2007, NIPS.

[37]  Ali Borji,et al.  Analysis of Scores, Datasets, and Models in Visual Saliency Prediction , 2013, 2013 IEEE International Conference on Computer Vision.

[38]  Meng Wang,et al.  Light field saliency vs. 2D saliency: A comparative study , 2015, Neurocomputing.

[39]  Gert Kootstra,et al.  Predicting Eye Fixations on Complex Visual Stimuli Using Local Symmetry , 2011, Cognitive Computation.

[40]  J.-L. Wu,et al.  Video Adaptation for Small Display Based on Content Recomposition , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[41]  Andriana Olmos,et al.  A biologically inspired algorithm for the recovery of shading and reflectance images , 2004 .

[42]  David A. Clausi,et al.  Statistical Textural Distinctiveness for Salient Region Detection in Natural Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Zhaoping Li,et al.  Feature-specific interactions in salience from combined feature contrasts: evidence for a bottom-up saliency map in V1. , 2007, Journal of vision.

[44]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[45]  Ying Wu,et al.  A unified approach to salient object detection via low rank matrix recovery , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Dongyue Chen,et al.  Scale-Invariant Amplitude Spectrum Modulation for Visual Saliency Detection , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[47]  Yao Li,et al.  Contextual Hypergraph Modeling for Salient Object Detection , 2013, 2013 IEEE International Conference on Computer Vision.

[48]  Aidong Zhang,et al.  Semantics-Based Image Retrieval by Region Saliency , 2002, CIVR.

[49]  D. Heeger,et al.  The Normalization Model of Attention , 2009, Neuron.

[50]  Jian Sun,et al.  Saliency Optimization from Robust Background Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Nuno Vasconcelos,et al.  Bottom-up saliency is a discriminant process , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[52]  Zi Huang,et al.  Tag localization with spatial correlations and joint group sparsity , 2011, CVPR 2011.

[53]  Tim K Marks,et al.  SUN: A Bayesian framework for saliency using natural statistics. , 2008, Journal of vision.

[54]  Meng Wang,et al.  Saliency Detection with a Deeper Investigation of Light Field , 2015, IJCAI.

[55]  L. Zhaoping,et al.  A theory of a saliency map in primary visual cortex (V1) tested by psychophysics of colour–orientation interference in texture segmentation , 2006 .

[56]  Huicheng Zheng,et al.  Fast-Learning Adaptive-Subspace Self-Organizing Map: An Application to Saliency-Based Invariant Image Feature Construction , 2008, IEEE Transactions on Neural Networks.

[57]  Wolfgang Effelsberg,et al.  Analysis of Disparity Maps for Detecting Saliency in Stereoscopic Video , 2013 .

[58]  Xiaochun Cao,et al.  Image aesthetics enhancement using composition-based saliency detection , 2014, Multimedia Systems.

[59]  Esa Rahtu,et al.  Fast and Efficient Saliency Detection Using Sparse Sampling and Kernel Density Estimation , 2011, SCIA.

[60]  J. Mollon,et al.  A neural basis for unique hues? , 2009, Current Biology.

[61]  Yang Liu,et al.  Interpolation-tuned salient region detection , 2013, Science China Information Sciences.

[62]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[63]  D. Heeger,et al.  Cross-orientation suppression in human visual cortex. , 2011, Journal of neurophysiology.

[64]  Ling Shao,et al.  Weakly-Supervised Cross-Domain Dictionary Learning for Visual Recognition , 2014, International Journal of Computer Vision.

[65]  Zhaoping Li A saliency map in primary visual cortex , 2002, Trends in Cognitive Sciences.

[66]  R. Shapley,et al.  Color in the Cortex: single- and double-opponent cells , 2011, Vision Research.

[67]  Xuelong Li,et al.  Saliency Detection by Multiple-Instance Learning , 2013, IEEE Transactions on Cybernetics.

[68]  Christof Koch,et al.  Image Signature: Highlighting Sparse Salient Regions , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[70]  Thomas Serre,et al.  A New Biologically Inspired Color Image Descriptor , 2012, ECCV.

[71]  King Ngi Ngan,et al.  Unsupervised extraction of visual attention objects in color images , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[72]  Frédo Durand,et al.  A Benchmark of Computational Models of Saliency to Predict Human Fixations , 2012 .

[73]  Ramon Baldrich,et al.  Saliency of color image derivatives: a comparison between computational models and human perception. , 2010, Journal of the Optical Society of America. A, Optics, image science, and vision.

[74]  Sabine Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[75]  Wen-Huang Cheng,et al.  A Visual Attention Based Region-of-Interest Determination Framework for Video Sequences , 2005, IEICE Trans. Inf. Syst..

[76]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[77]  Shijian Lu,et al.  Robust and Efficient Saliency Modeling from Image Co-Occurrence Histograms , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[78]  John K. Tsotsos,et al.  Saliency Based on Information Maximization , 2005, NIPS.

[79]  Christof Koch,et al.  Learning visual saliency by combining feature maps in a nonlinear manner using AdaBoost. , 2012, Journal of vision.

[80]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[81]  Lei Guo,et al.  Saliency detection based on feature learning using Deep Boltzmann Machines , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[82]  Junwei Han,et al.  Efficient, simultaneous detection of multi-class geospatial targets based on visual saliency modeling and discriminative learning of sparse coding , 2014 .

[83]  D. Hubel,et al.  Anatomy and physiology of a color system in the primate visual cortex , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[84]  T. Ravindra Babu,et al.  Scalable sequential alternating proximal methods for sparse structural SVMs and CRFs , 2013, Knowledge and Information Systems.

[85]  Liqing Zhang,et al.  Dynamic visual attention: searching for coding length increments , 2008, NIPS.

[86]  M. Carandini,et al.  Summation and division by neurons in primate visual cortex. , 1994, Science.