Bottom-up saliency model generation using superpixels

Prediction of human visual attention is more and more frequently applicable in computer graphics, image processing, human-computer interaction and computer vision. Human attention is influenced by various bottom-up stimuli such as colour, intensity and orientation as well as top-down stimuli related to our memory. Saliency models implement bottom-up factors of visual attention and represent the conspicuousness of a given environment using a saliency map. In general, visual attention processing consists of identification of individual features and their subsequent combination to perceive whole objects. Standard hierarchical saliency methods do not respect the shape of objects and model the saliency as the pixel-by-pixel difference between the centre and its surround. The aim of our work is to improve the saliency prediction using a superpixel-based approach whose regions should correspond to objects borders. In this paper we propose a novel saliency method that combines a hierarchical processing of visual features and a superpixel-based segmentation. The proposed method is compared with existing saliency models and evaluated on a publicly available dataset.

[1]  S. Süsstrunk,et al.  SLIC Superpixels ? , 2010 .

[2]  Peer Neubert,et al.  Superpixel Benchmark and Comparison , 2012 .

[3]  Truong Q. Nguyen,et al.  Video processing with scale-aware saliency: Application to Frame Rate Up-Conversion , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Nicolas Riche,et al.  Saliency and Human Fixations: State-of-the-Art and Study of Comparison Metrics , 2013, 2013 IEEE International Conference on Computer Vision.

[5]  John K. Tsotsos,et al.  Saliency Based on Information Maximization , 2005, NIPS.

[6]  Sílvio Filipe,et al.  RETRACTED ARTICLE: From the human visual system to the computational models of visual attention: a survey , 2015, Artificial Intelligence Review.

[7]  Sabine Süsstrunk,et al.  Saliency detection for content-aware image resizing , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[8]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Laurent Itti,et al.  Biologically Inspired Mobile Robot Vision Localization , 2009, IEEE Transactions on Robotics.

[10]  A. L. I︠A︡rbus Eye Movements and Vision , 1967 .

[11]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Shaogang Gong,et al.  Salient motion detection in crowded scenes , 2012, 2012 5th International Symposium on Communications, Control and Signal Processing.

[13]  C. Scheier,et al.  Visual attention in a mobile robot , 1997, ISIE '97 Proceeding of the IEEE International Symposium on Industrial Electronics.

[14]  Wanda Benesova,et al.  Fast Superpixel Segmentation Using Morphological Processing , 2014 .

[15]  John K. Tsotsos,et al.  An Information Theoretic Model of Saliency and Visual Search , 2008, WAPCV.

[16]  John K. Tsotsos,et al.  Saliency, attention, and visual search: an information theoretic approach. , 2009, Journal of vision.

[17]  P. Le Callet,et al.  SELECTIVE H . 264 VIDEO CODING BASED ON A SALIENCY MAP , 2005 .

[18]  Kenneth Holmqvist,et al.  Eye tracking: a comprehensive guide to methods and measures , 2011 .

[19]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[20]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.

[21]  Ramesh Raskar,et al.  Automatic image retargeting , 2005, MUM '05.

[22]  Lihi Zelnik-Manor,et al.  Context-aware saliency detection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Bohyung Han,et al.  Generalized Background Subtraction Using Superpixels with Label Integrated Motion Estimation , 2014, ECCV.

[24]  Patrick Le Callet,et al.  Visual Attention and Applications in Multimedia Technologies , 2013, Proceedings of the IEEE.

[25]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[26]  M. Narasimha Murty,et al.  Pattern Recognition - An Algorithmic Approach , 2011, Undergraduate Topics in Computer Science.

[27]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Tim K Marks,et al.  SUN: A Bayesian framework for saliency using natural statistics. , 2008, Journal of vision.

[29]  Thierry Baccino,et al.  Methods for comparing scanpaths and saliency maps: strengths and weaknesses , 2012, Behavior Research Methods.

[30]  Justus H. Piater,et al.  Closed-Loop Learning of Visual Control Policies , 2011, J. Artif. Intell. Res..

[31]  Nuno Vasconcelos,et al.  Decision-Theoretic Saliency: Computational Principles, Biological Plausibility, and Implications for Neurophysiology and Psychophysics , 2009, Neural Computation.

[32]  Carla H. Lagorio,et al.  Psychology , 1929, Nature.

[33]  Bernhard Schölkopf,et al.  Center-surround patterns emerge as optimal predictors for human saccade targets. , 2009, Journal of vision.

[34]  Martin A. Giese,et al.  Hooligan detection: The effects of saliency and expert knowledge , 2011, ICDP.

[35]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[36]  Stefano Soatto,et al.  Quick Shift and Kernel Methods for Mode Seeking , 2008, ECCV.

[37]  Luc Vincent,et al.  Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Antonio Torralba,et al.  Top-down control of visual attention in object detection , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[39]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[40]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[42]  Stefan Schaal,et al.  Overt visual attention for a humanoid robot , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[43]  Dattaguru V Kamat A framework for visual saliency detection with applications to image thumbnailing , 2009 .

[44]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[45]  Patrick Le Callet,et al.  What we see is most likely to be what matters: Visual attention and applications , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[46]  Albert Ali Salah,et al.  A Selective Attention-Based Method for Visual Pattern Recognition with Application to Handwritten Digit Recognition and Face Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Nuno Vasconcelos,et al.  On the plausibility of the discriminant center-surround hypothesis for visual saliency. , 2008, Journal of vision.

[48]  Sven J. Dickinson,et al.  TurboPixels: Fast Superpixels Using Geometric Flows , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Laurent Itti,et al.  Automatic foveation for video compression using a neurobiological model of visual attention , 2004, IEEE Transactions on Image Processing.

[50]  Ethem Alpaydin,et al.  Introduction to machine learning , 2004, Adaptive computation and machine learning.

[51]  H. Bastian Sensation and Perception.—I , 1869, Nature.

[52]  Nuno Vasconcelos,et al.  Discriminant Saliency for Visual Recognition from Cluttered Scenes , 2004, NIPS.

[53]  L. King,et al.  The Science of Psychology: An Appreciative View , 2007 .

[54]  Ewald Hering,et al.  Grundzüge der Lehre vom Lichtsinn. , 1920 .

[55]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Fred Stentiford,et al.  Attention Based Auto Image Cropping , 2007, ICVS 2007.

[57]  智一 吉田,et al.  Efficient Graph-Based Image Segmentationを用いた圃場図自動作成手法の検討 , 2014 .