Motion saliency outweighs other low-level features while watching videos

The importance of motion in attracting attention is well known. While watching videos, where motion is prevalent, how do we quantify the regions that are motion salient? In this paper, we investigate the role of motion in attention and compare it with the influence of other low-level features like image orientation and intensity. We propose a framework for motion saliency. In particular, we integrate motion vector information with spatial and temporal coherency to generate a motion attention map. The results show that our model achieves good performance in identifying regions that are moving and salient. We also find motion to have greater influence on saliency than other low-level features when watching videos.

[1]  P Reinagel,et al.  Natural scene statistics at the centre of gaze. , 1999, Network.

[2]  Joseph H. Goldberg,et al.  Identifying fixations and saccades in eye-tracking protocols , 2000, ETRA.

[3]  Laurent Itti,et al.  Automatic foveation for video compression using a neurobiological model of visual attention , 2004, IEEE Transactions on Image Processing.

[4]  Laurent Itti,et al.  The role of memory in guiding attention during natural vision. , 2006, Journal of vision.

[5]  Christoph Kayser,et al.  Fixations in natural scenes: Interaction of image structure and image content , 2006, Vision Research.

[6]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[7]  Michael L. Mack,et al.  VISUAL SALIENCY DOES NOT ACCOUNT FOR EYE MOVEMENTS DURING VISUAL SEARCH IN REAL-WORLD SCENES , 2007 .

[8]  Eileen Kowler,et al.  Shapes, surfaces and saccades , 1999, Vision Research.

[9]  Mriganka Sur,et al.  Image Structure at the Center of Gaze during Free Viewing , 2006, Journal of Cognitive Neuroscience.

[10]  Eero P. Simoncelli,et al.  A model of neuronal responses in visual area MT , 1998, Vision Research.

[11]  Pierre Baldi,et al.  A principled approach to detecting surprising events in video , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[12]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[13]  P. König,et al.  Differences of monkey and human overt attention under natural conditions , 2006, Vision Research.

[14]  Iain D. Gilchrist,et al.  Visual correlates of fixation selection: effects of scale and time , 2005, Vision Research.

[15]  Kai-Kuang Ma,et al.  Adaptive rood pattern search for fast block-matching motion estimation , 2002, IEEE Trans. Image Process..

[16]  Eileen Kowler,et al.  Localization of shapes: eye movements and perception compared , 2003, Vision Research.

[17]  A. L. I︠A︡rbus.  Eye Movements and Vision , 1967 .

[18]  Derrick J. Parkhurst,et al.  Texture contrast attracts overt visual attention in natural scenes , 2004, The European journal of neuroscience.

[19]  R. Abrams,et al.  Motion Onset Captures Attention , 2003, Psychological science.

[20]  Thomas Martinetz,et al.  Guiding the mind's eye: improving communication and vision by external control of the scanpath , 2006, Electronic Imaging.