Gaze allocation in natural stimuli: Comparing free exploration to head-fixed viewing conditions

“Natural” gaze is typically measured by tracking eye positions during scene presentation in laboratory settings. How informative are such investigations for real-world conditions? Using a mobile eyetracking setup (“EyeSeeCam”), we measure gaze during free exploration of various in- and outdoor environments, while simultaneously recording head-centred videos. Here, we replay these videos in a laboratory setup. Half of the laboratory observers view the movies continuously, half as sequences of static 1-second frames. We find a bias of eye position to the stimulus centre, which is strongest in the 1 s frame replay condition. As a consequence, interobserver consistency is highest in this condition, though not fully explained by spatial bias alone. This leaves room for image specific bottom-up models to predict gaze beyond generic biases. Indeed, the “saliency map” predicts eye position in all conditions, and best for continuous replay. Continuous replay predicts real-world gaze better than 1 s frame replay does. In conclusion, experiments and models benefit from preserving the spatial statistics and temporal continuity of natural stimuli to improve their validity for real-world gaze behaviour.

[1]  R. C. Langford.  How People Look at Pictures, A Study of the Psychology of Perception in Art. , 1936 .

[2]  A. L. I︠A︡rbus.  Eye Movements and Vision , 1967 .

[3]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[4]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[5]  David N. Lee,et al.  Where we look when we steer , 1994, Nature.

[6]  D. S. Wooding,et al.  The relationship between the locations of spatial features and those of fixations made during visual examination of briefly presented images. , 1996, Spatial vision.

[7]  V. Tosi,et al.  Scanning eye movements made when viewing film: preliminary observations. , 1997, The International journal of neuroscience.

[8]  E. Rolls,et al.  INVARIANT FACE AND OBJECT RECOGNITION IN THE VISUAL SYSTEM , 1997, Progress in Neurobiology.

[9]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[10]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[11]  M. Land,et al.  The Roles of Vision and Eye Movements in the Control of Activities of Daily Living , 1998, Perception.

[12]  P Reinagel,et al.  Natural scene statistics at the centre of gaze. , 1999, Network.

[13]  J. Stahl,et al.  Amplitude of human head movements associated with horizontal saccades , 1999, Experimental Brain Research.

[14]  Michael F. Land,et al.  From eye movements to actions: how batsmen hit the ball , 2000, Nature Neuroscience.

[15]  G. Hauske,et al.  Object and scene analysis by saccadic eye-movements: an investigation with higher-order statistics. , 2000, Spatial vision.

[16]  Claudio M. Privitera,et al.  Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[18]  B. Tatler,et al.  Steering with the head The visual strategy of a racing driver , 2001, Current Biology.

[19]  M. Hayhoe,et al.  In what ways do eye movements contribute to everyday activities? , 2001, Vision Research.

[20]  Is Your Eye on the Ball ? : Eye Tracking Golfers while Putting , 2001 .

[21]  Jochen Triesch,et al.  Vision in natural and virtual environments , 2002, ETRA.

[22]  Frans W Cornelissen,et al.  The Eyelink Toolbox: Eye tracking with MATLAB and the Psychophysics Toolbox , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[23]  Derrick J. Parkhurst,et al.  Modeling the role of salience in the allocation of overt visual attention , 2002, Vision Research.

[24]  Dana H. Ballard,et al.  Eye Movements for Reward Maximization , 2003, NIPS.

[25]  P. König,et al.  Does luminance‐contrast contribute to a saliency map for overt visual attention? , 2003, The European journal of neuroscience.

[26]  J. Fuller,et al.  Head movement propensity , 2004, Experimental Brain Research.

[27]  L. Itti,et al.  Modeling the influence of task on attention , 2005, Vision Research.

[28]  L. Itti.  Author address: , 1999 .

[29]  Julian Eggert,et al.  Learning viewpoint invariant object representations using a temporal coherence principle , 2005, Biological Cybernetics.

[30]  Asha Iyer,et al.  Components of bottom-up gaze allocation in natural images , 2005, Vision Research.

[31]  Iain D. Gilchrist,et al.  Visual correlates of fixation selection: effects of scale and time , 2005, Vision Research.

[32]  Mary Hayhoe,et al.  The role of internal models and prediction in catching balls , 2005, AAAI 2005.

[33]  Klaus Bartl,et al.  Eye movement driven head-mounted camera: it looks where the eyes look , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[34]  T. Brandt,et al.  Documentation and teaching of surgery with an eye movement driven head-mounted camera: see what the surgeon sees and does. , 2006, Studies in health technology and informatics.

[35]  Roland J. Baddeley,et al.  High frequency edges (but not contrast) predict where we fixate: A Bayesian system identification analysis , 2006, Vision Research.

[36]  Mary Hayhoe,et al.  Control of attention and gaze in complex environments. , 2006, Journal of vision.

[37]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[38]  Christoph Kayser,et al.  Fixations in natural scenes: Interaction of image structure and image content , 2006, Vision Research.

[39]  L. Itti,et al.  Visual causes versus correlates of attentional selection in dynamic scenes , 2006, Vision Research.

[40]  Simon J. Thorpe,et al.  Ultra-rapid object detection with saccadic eye movements: Visual processing speed revisited , 2006, Vision Research.

[41]  T. Brandt,et al.  A third eye for the surgeon , 2006, Journal of Neurology, Neurosurgery & Psychiatry.

[42]  Peter König,et al.  Human eye-head co-ordination in natural exploration , 2007, Network.

[43]  Dana H. Ballard,et al.  Modeling embodied visual behaviors , 2007, TAP.

[44]  Mary M Hayhoe,et al.  Task and context determine where you look. , 2016, Journal of vision.

[45]  Christof Koch,et al.  Predicting human gaze using low-level saliency combined with face detection , 2007, NIPS.

[46]  Benjamin W Tatler,et al.  The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. , 2007, Journal of vision.

[47]  Michael L. Mack,et al.  VISUAL SALIENCY DOES NOT ACCOUNT FOR EYE MOVEMENTS DURING VISUAL SEARCH IN REAL-WORLD SCENES , 2007 .

[48]  Klaus Bartl,et al.  The combination of a mobile gaze-driven and a head-mounted camera in a Hybrid perspective setup , 2007, 2007 IEEE International Conference on Systems, Man and Cybernetics.

[49]  Peter König,et al.  Salient features in gaze-aligned recordings of human visual input during free exploration of natural environments. , 2008, Journal of vision.

[50]  Laurent Itti,et al.  Applying computational tools to predict gaze direction in interactive visual environments , 2008, TAP.

[51]  Laurent Itti,et al.  Interesting objects are visually salient. , 2008, Journal of vision.

[52]  L. Zhaoping.  Attention capture by eye of origin singletons even without awareness--a hallmark of a bottom-up saliency map in the primary visual cortex. , 2008, Journal of vision.

[53]  C. Koch,et al.  Task-demands can immediately reverse the effects of sensory-driven saliency in complex visual stimuli. , 2008, Journal of vision.

[54]  P. Perona,et al.  Objects predict fixations better than early saliency. , 2008, Journal of vision.

[55]  Tim K Marks,et al.  SUN: A Bayesian framework for saliency using natural statistics. , 2008, Journal of vision.

[56]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[57]  Stefan Kohlbecher,et al.  EyeSeeCam: An Eye Movement–Driven Head Camera for the Examination of Natural Visual Exploration , 2009, Annals of the New York Academy of Sciences.

[58]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.

[59]  Peter König,et al.  Distinct Roles for Eye and Head Movements in Selecting Salient Image Parts during Natural Exploration , 2009, Annals of the New York Academy of Sciences.

[60]  Peter König,et al.  Eye–Head Coordination during Free Exploration in Human and Cat , 2009, Annals of the New York Academy of Sciences.

[61]  Jeff B. Pelz,et al.  Predictive eye movements in squash , 2010 .