Clustering of Gaze During Dynamic Scene Viewing is Predicted by Motion

Where does one attend when viewing dynamic scenes? Research into the factors influencing gaze location during static scene viewing have reported that low-level visual features contribute very little to gaze location especially when opposed by high-level factors such as viewing task. However, the inclusion of transient features such as motion in dynamic scenes may result in a greater influence of visual features on gaze allocation and coordination of gaze across viewers. In the present study, we investigated the contribution of low- to mid-level visual features to gaze location during free-viewing of a large dataset of videos ranging in content and length. Signal detection analysis on visual features and Gaussian Mixture Models for clustering gaze was used to identify the contribution of visual features to gaze location. The results show that mid-level visual features including corners and orientations can distinguish between actual gaze locations and a randomly sampled baseline. However, temporal features such as flicker, motion, and their respective contrasts were the most predictive of gaze location. Additionally, moments in which all viewers’ gaze tightly clustered in the same location could be predicted by motion. Motion and mid-level visual features may influence gaze allocation in dynamic scenes, but it is currently unclear whether this influence is involuntary or due to correlations with higher order factors such as scene semantics.

[1]  A. Mizuno,et al.  A change of the leading player in flow Visualization technique , 2006, J. Vis..

[2]  J. Findlay,et al.  Active Vision: The Psychology of Looking and Seeing , 2003 .

[3]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[4]  E. B. Andersen,et al.  Information Science and Statistics , 1986 .

[5]  J. Henderson,et al.  Overt attentional prioritization of new objects and feature changes during real-world scene viewing , 2009 .

[6]  R. Baddeley,et al.  Searching for filters with 'interesting' output distributions: an uninteresting direction to explore? , 1996, Network.

[7]  Roland J. Baddeley,et al.  High frequency edges (but not contrast) predict where we fixate: A Bayesian system identification analysis , 2006, Vision Research.

[8]  Michael L. Mack,et al.  VISUAL SALIENCY DOES NOT ACCOUNT FOR EYE MOVEMENTS DURING VISUAL SEARCH IN REAL-WORLD SCENES , 2007 .

[9]  C Blakemore,et al.  On the existence of neurones in the human visual system selectively sensitive to the orientation and size of retinal images , 1969, The Journal of physiology.

[10]  David Bordwell,et al.  Film Art: An Introduction , 1979 .

[11]  Iain D Gilchrist,et al.  Oculomotor capture by transient events: a comparison of abrupt onsets, offsets, motion, and flicker. , 2008, Journal of vision.

[12]  M. Sanders.  Handbook of Sensory Physiology , 1975 .

[13]  Zenzi M. Griffin,et al.  Why Look? Reasons for Eye Movements Related to Language Production. , 2004 .

[14]  L. Itti.  Author address: , 1999 .

[15]  M. Hayhoe,et al.  In what ways do eye movements contribute to everyday activities? , 2001, Vision Research.

[16]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[17]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[18]  Pierre Baldi,et al.  A principled approach to detecting surprising events in video , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[19]  Paul L. Rosin.  A simple method for detecting salient regions , 2009, Pattern Recognit..

[20]  Marcus Nyström,et al.  Effect of compressed offline foveated video on viewing behavior and subjective quality , 2010, TOMCCAP.

[21]  David N. Lee,et al.  Where we look when we steer , 1994, Nature.

[22]  Gerhard Krieger,et al.  Investigation of a sensorimotor system for saccadic scene analysis: an integrated approach , 1998 .

[23]  Laurent Itti,et al.  The role of memory in guiding attention during natural vision. , 2006, Journal of vision.

[24]  Derrick J. Parkhurst,et al.  Modeling the role of salience in the allocation of overt visual attention , 2002, Vision Research.

[25]  G. Mather,et al.  Two Channels for Flicker in the Human Visual System , 1984, Perception.

[26]  B. Moulden,et al.  The Standard Deviation of Luminance as a Metric for Contrast in Random-Dot Images , 1990, Perception.

[27]  Michael C. Frank,et al.  Development of infants’ attention to faces during the first year , 2009, Cognition.

[28]  Carrick C. Williams,et al.  Eye movements and picture processing during recognition , 2003, Perception & psychophysics.

[29]  T. Foulsham,et al.  How Does the Purpose of Inspection Influence the Potency of Visual Salience in Scene Perception? , 2007, Perception.

[30]  J. Alison Noble,et al.  Finding Corners , 1988, Alvey Vision Conference.

[31]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[32]  L. Itti.  Quantitative modelling of perceptual salience at human eye position , 2006 .

[33]  L. Rask.  ON APPARENT MOVEMENT , 1967 .

[34]  Cordelia Schmid,et al.  Evaluation of Interest Point Detectors , 2000, International Journal of Computer Vision.

[35]  S. Yantis,et al.  Stimulus-driven attentional capture: evidence from equiluminant visual objects. , 1994, Journal of experimental psychology. Human perception and performance.

[36]  Robin L. Hill,et al.  Eye movements : a window on mind and brain , 2007 .

[37]  R. Rosenholtz.  A simple saliency model predicts a number of motion popout phenomena , 1999, Vision Research.

[38]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[39]  Jeffrey M. Zacks,et al.  Human brain activity time-locked to perceptual event boundaries , 2001, Nature Neuroscience.

[40]  T. Smith.  An Attentional Theory of Continuity Editing , 2006 .

[41]  J. Wolfe,et al.  What attributes guide the deployment of visual attention and how do they do it? , 2004, Nature Reviews Neuroscience.

[42]  Eli Peli,et al.  Where people look when watching movies: Do all viewers look at the same place? , 2007, Comput. Biol. Medicine.

[43]  Keith Baker,et al.  5th Alvey vision Conference , 1990, Image Vis. Comput..

[44]  Iain D. Gilchrist,et al.  Visual correlates of fixation selection: effects of scale and time , 2005, Vision Research.

[45]  T. Foulsham,et al.  What can saliency models predict about eye movements? Spatial and sequential aspects of fixations during encoding and recognition. , 2008, Journal of vision.

[46]  Robert B. Fisher,et al.  A computer vision model for visual-object-based attention and eye movements , 2008, Comput. Vis. Image Underst..

[47]  Derrick J. Parkhurst,et al.  Scene content selected by active vision. , 2003, Spatial vision.

[48]  Rajesh P. N. Rao,et al.  Eye movements in iconic visual search , 2002, Vision Research.

[49]  Benjamin W Tatler,et al.  The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. , 2007, Journal of vision.

[50]  Eyal M. Reingold,et al.  Area activation: a computational model of saccadic selectivity in visual search , 2003, Cogn. Sci..

[51]  Derrick J. Parkhurst,et al.  Texture contrast attracts overt visual attention in natural scenes , 2004, The European journal of neuroscience.

[52]  D. S. Wooding,et al.  Automatic control of saccadic eye movements made in visual inspection of briefly presented 2-D images. , 1995, Spatial vision.

[53]  Olaf Blanke,et al.  Gravity and observer's body orientation influence the visual perception of human body postures. , 2009, Journal of vision.

[54]  Claudio M. Privitera,et al.  Human-vision-based selection of image processing algorithms for planetary exploration , 2003, IEEE Trans. Image Process..

[55]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[56]  D. S. Wooding,et al.  Fixation sequences made during visual examination of briefly presented 2D images. , 1997, Spatial vision.

[57]  R. Baddeley,et al.  Do we look at lights? Using mixture modelling to distinguish between low- and high-level factors in natural image viewing , 2009 .

[58]  P. Bex,et al.  Spatial frequency, phase, and the contrast of natural images. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[59]  Richard Stevens,et al.  Are you seeing what I'm seeing? An eye-tracking evaluation of dynamic scenes , 2009, Digit. Creativity.

[60]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[61]  C. Latimer,et al.  Eye-movement data: Cumulative fixation time and cluster analysis , 1988 .

[62]  Nobuyuki Hiruma,et al.  Determining comprehension and quality of TV programs using eye-gaze tracking , 2008, Pattern Recognit..

[63]  Brian J. Murphy,et al.  Pattern thresholds for moving and stationary gratings during smooth eye movement , 1978, Vision Research.

[64]  Xin Chen,et al.  Real-world visual search is dominated by top-down guidance , 2006, Vision Research.

[65]  J. Theeuwes.  Abrupt luminance change pops out; abrupt color change does not , 1995, Perception & psychophysics.

[66]  L. Itti,et al.  Quantifying center bias of observers in free viewing of dynamic natural scenes. , 2009, Journal of vision.

[67]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[68]  Hans P. Moravec.  Obstacle avoidance and navigation in the real world by a seeing robot rover , 1980 .

[69]  Robert A. Marino,et al.  Free viewing of dynamic stimuli by humans and monkeys. , 2009, Journal of vision.

[70]  O. Meur,et al.  Predicting visual fixations on video based on low-level visual features , 2007, Vision Research.

[71]  Jang-Kyoo Shin,et al.  Biologically Inspired Saliency Map Model for Bottom-up Visual Attention , 2002, Biologically Motivated Computer Vision.

[72]  P Reinagel,et al.  Natural scene statistics at the centre of gaze. , 1999, Network.

[73]  J. Henderson,et al.  Prioritization of new objects in real-world scenes: evidence from eye movements. , 2005, Journal of experimental psychology. Human perception and performance.

[74]  F. Ohl,et al.  Fallacies in behavioural interpretation of auditory cortex plasticity , 2004, Nature Reviews Neuroscience.

[75]  Marcus Nyström,et al.  Semantic override of low-level features in image viewing - both initially and overall , 2008 .

[76]  D. Heeger,et al.  Neurocinematics: The Neuroscience of Film , 2008 .

[77]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[78]  S. Yantis,et al.  Abrupt visual onsets and selective attention: evidence from visual search. , 1984, Journal of experimental psychology. Human perception and performance.

[79]  M. Carandini,et al.  Summation and division by neurons in primate visual cortex. , 1994, Science.

[80]  Michael F. Land,et al.  From eye movements to actions: how batsmen hit the ball , 2000, Nature Neuroscience.

[81]  D. S. Wooding,et al.  The relationship between the locations of spatial features and those of fixations made during visual examination of briefly presented images. , 1996, Spatial vision.

[82]  John M. Henderson,et al.  Attentional synchrony in static and dynamic scenes , 2010 .

[83]  D. Simons.  Attentional capture and inattentional blindness , 2000, Trends in Cognitive Sciences.

[84]  Claudio M. Privitera,et al.  Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[85]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[86]  Antonio Torralba,et al.  Statistics of natural image categories , 2003, Network.

[87]  J. Henderson,et al.  Object appearance, disappearance, and attention prioritization in real-world scenes , 2005, Psychonomic bulletin & review.

[88]  A. L. I︠A︡rbus.  Eye Movements and Vision , 1967 .

[89]  Jeffery. M. Zacks,et al.  Activation of human motion processing areas during event perception , 2003, Cognitive, affective & behavioral neuroscience.

[90]  V. Tosi,et al.  Scanning eye movements made when viewing film: preliminary observations. , 1997, The International journal of neuroscience.

[91]  J. Henderson.  Regarding Scenes , 2007 .

[92]  H. Barlow.  Vision Science: Photons to Phenomenology by Stephen E. Palmer , 2000, Trends in Cognitive Sciences.

[93]  P. Perona,et al.  Objects predict fixations better than early saliency. , 2008, Journal of vision.

[94]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[95]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[96]  P. König,et al.  Does luminance‐contrast contribute to a saliency map for overt visual attention? , 2003, The European journal of neuroscience.

[97]  G. Hauske,et al.  Object and scene analysis by saccadic eye-movements: an investigation with higher-order statistics. , 2000, Spatial vision.

[98]  Katsumi Aoki,et al.  Recent development of flow visualization , 2004, J. Vis..

[99]  Marcus Nyström,et al.  Variable resolution images and their effects on eye movements during free viewing , 2007, Electronic Imaging.

[100]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[101]  Mark Wexler,et al.  The nonlinear structure of motion perception during smooth eye movements. , 2009, Journal of vision.

[102]  R. C. Langford.  How People Look at Pictures, A Study of the Psychology of Perception in Art. , 1936 .

[103]  Nicolai Petkov,et al.  BIOLOGICALLY MOTIVATED COMPUTER VISION, PROCEEDINGS , 2002 .

[104]  P. König,et al.  Gaze allocation in natural stimuli: Comparing free exploration to head-fixed viewing conditions , 2009 .

[105]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[106]  Jean M. Vettel,et al.  Visual motion and the neural correlates of event perception , 2006, Brain Research.

[107]  Aude Billard,et al.  From Animals to Animats , 2004 .

[108]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[109]  S. Yantis.  Control of visual attention. , 1998 .

[110]  F. Attneave.  Some informational aspects of visual perception. , 1954, Psychological review.

[111]  G. Zelinsky.  A theory of eye movements during target acquisition. , 2008, Psychological review.

[112]  L. Itti,et al.  Visual causes versus correlates of attentional selection in dynamic scenes , 2006, Vision Research.

[113]  Tomaso A. Poggio,et al.  On Edge Detection , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[114]  S. Ullman.  The Interpretation of Visual Motion , 1979 .

[115]  Philip J. Barnard,et al.  Using Film Cutting Techniques in Interface Design , 2003, Hum. Comput. Interact..

[116]  Douglas DeCarlo,et al.  Robust clustering of eye movement recordings for quantification of visual interest , 2004, ETRA.

[117]  I. Rentschler,et al.  Intrinsic two-dimensional features as textons. , 1998, Journal of the Optical Society of America. A, Optics, image science, and vision.

[118]  George L. Malcolm,et al.  Searching in the dark: Cognitive relevance drives attention in real-world scenes , 2009, Psychonomic bulletin & review.

[119]  L. Itti,et al.  Modeling the influence of task on attention , 2005, Vision Research.

[120]  S. Anstis.  The perception of apparent movement. , 1980, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[121]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[122]  Hao Sun,et al.  The temporal properties of the response of macaque ganglion cells and central mechanisms of flicker detection. , 2007, Journal of vision.

[123]  Wa James Tam,et al.  Static and dynamic spatial resolution in image coding: an investigation of eye movements , 1991, Electronic Imaging.

[124]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[125]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[126]  Roland J. Baddeley,et al.  The nature of the visual representations involved in eye movements when walking down the street , 2009 .

[127]  C. Koch,et al.  Attention activates winner-take-all competition among visual filters , 1999, Nature Neuroscience.