Task and context determine where you look.

The deployment of human gaze has been almost exclusively studied independent of any specific ongoing task and limited to two-dimensional picture viewing. This contrasts with its use in everyday life, which mostly consists of purposeful tasks where gaze is crucially involved. To better understand deployment of gaze under such circumstances, we devised a series of experiments, in which subjects navigated along a walkway in a virtual environment and executed combinations of approach and avoidance tasks. The position of the body and the gaze were monitored during the execution of the task combinations and dependence of gaze on the ongoing tasks as well as the visual features of the scene was analyzed. Gaze distributions were compared to a random gaze allocation strategy as well as a specific "saliency model." Gaze distributions showed high similarity across subjects. Moreover, the precise fixation locations on the objects depended on the ongoing task to the point that the specific tasks could be predicted from the subject's fixation data. By contrast, gaze allocation according to a random or a saliency model did not predict the executed fixations or the observed dependence of fixation locations on the specific task.

[1]  U. Neisser.  VISUAL SEARCH. , 1964, Scientific American.

[2]  A. L. I︠A︡rbus.  Eye Movements and Vision , 1967 .

[3]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[4]  H. Barlow.  Vision: A computational investigation into the human representation and processing of visual information: David Marr. San Francisco: W. H. Freeman, 1982. pp. xvi + 397 , 1983 .

[5]  M. Posner,et al.  Components of visual orienting , 1984 .

[6]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[7]  Dana H. Ballard,et al.  Animate Vision , 1991, Artif. Intell..

[8]  E Kowler,et al.  Saccadic localization of eccentric forms. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[9]  J. C. Johnston,et al.  Involuntary attentional capture by abrupt onsets , 1992, Perception & psychophysics.

[10]  S. Yantis,et al.  Visual motion and attentional capture , 1994, Perception & psychophysics.

[11]  David N. Lee,et al.  Where we look when we steer , 1994, Nature.

[12]  Julie C. Sedivy,et al.  Subject Terms: Linguistics Language Eyes & eyesight Cognition & reasoning , 1995 .

[13]  D. Ballard,et al.  Memory Representations in Natural Tasks , 1995, Journal of Cognitive Neuroscience.

[14]  B. Dosher,et al.  The role of attention in the programming of saccades , 1995, Vision Research.

[15]  D. S. Wooding,et al.  The relationship between the locations of spatial features and those of fixations made during visual examination of briefly presented images. , 1996, Spatial vision.

[16]  S. Shimojo,et al.  The Preparation of Goal-Directed Movements Requires Selective Visual Attention: Evidence from the Line-Motion Illusion , 1997 .

[17]  W. Ehrenstein,et al.  Suprathreshold Motion Sensitivity and Ocular Dominance , 1997 .

[18]  Rajesh P. N. Rao,et al.  Embodiment is the foundation, not a level , 1996, Behavioral and Brain Sciences.

[19]  M. Chun,et al.  Contextual Cueing: Implicit Learning and Memory of Visual Context Guides Spatial Attention , 1998, Cognitive Psychology.

[20]  J. Henderson,et al.  High-level scene perception. , 1999, Annual review of psychology.

[21]  M. Land,et al.  The Roles of Vision and Eye Movements in the Control of Activities of Daily Living , 1998, Perception.

[22]  H. Deubel,et al.  Current oculomotor research : physiological and psychological aspects , 1999 .

[23]  P Reinagel,et al.  Natural scene statistics at the centre of gaze. , 1999, Network.

[24]  Heiner Deubel,et al.  Object Recognition and Goal-Directed Eye or Hand Movements are Coupled by Visual Attention , 1999 .

[25]  Michael F. Land,et al.  From eye movements to actions: how batsmen hit the ball , 2000, Nature Neuroscience.

[26]  G. Hauske,et al.  Object and scene analysis by saccadic eye-movements: an investigation with higher-order statistics. , 2000, Spatial vision.

[27]  Jeff B. Pelz,et al.  Portable eyetracking: a study of natural eye movements , 2000, Electronic Imaging.

[28]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[29]  S. Liversedge,et al.  Saccadic eye movements and cognition , 2000, Trends in Cognitive Sciences.

[30]  W. Schultz.  Multiple reward signals in the brain , 2000, Nature Reviews Neuroscience.

[31]  C. Koch,et al.  Models of bottom-up and top-down visual attention , 2000 .

[32]  A. Oliva,et al.  Diagnostic Colors Mediate Scene Recognition , 2000, Cognitive Psychology.

[33]  M. Hayhoe,et al.  What controls attention in natural environments? , 2001, Vision Research.

[34]  M. Hayhoe,et al.  In what ways do eye movements contribute to everyday activities? , 2001, Vision Research.

[35]  R. Johansson,et al.  Eye–Hand Coordination in Object Manipulation , 2001, The Journal of Neuroscience.

[36]  Paul R. Schrater,et al.  Perceiving visual expansion without optic flow , 2001, Nature.

[37]  A. Noë,et al.  A sensorimotor account of vision and visual consciousness. , 2001, The Behavioral and brain sciences.

[38]  William H. Warren,et al.  Optic flow is used to control human walking , 2001, Nature Neuroscience.

[39]  K. Gegenfurtner,et al.  The contributions of color to recognition memory for natural scenes. , 2002, Journal of experimental psychology. Learning, memory, and cognition.

[40]  Dana H. Ballard,et al.  Eye Movements for Reward Maximization , 2003, NIPS.

[41]  J. Henderson.  Human gaze control during real-world scene perception , 2003, Trends in Cognitive Sciences.

[42]  D. Ballard,et al.  What you see is what you need. , 2003, Journal of vision.

[43]  M. Land.  Eye Movements in Daily Life , 2003 .

[44]  Antonio Torralba,et al.  Top-down control of visual attention in object detection , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[45]  Mary M Hayhoe,et al.  Visual memory and motor planning in a natural task. , 2003, Journal of vision.

[46]  R. Johansson,et al.  Action plans used in action observation , 2003, Nature.

[47]  M. Landy,et al.  Statistical decision theory and trade-offs in the control of motor response. , 2003, Spatial vision.

[48]  P. König,et al.  Does luminance‐contrast contribute to a saliency map for overt visual attention? , 2003, The European journal of neuroscience.

[49]  P. Glimcher.  The neurobiology of visual-saccadic decision making. , 2003, Annual review of neuroscience.

[50]  Derrick J. Parkhurst,et al.  Scene content selected by active vision. , 2003, Spatial vision.

[51]  M. Hayhoe.  Advances in Relating Eye Movements and Cognition. , 2004, Infancy : the official journal of the International Society on Infant Studies.

[52]  C. Gilbert,et al.  Perceptual learning and top-down influences in primary visual cortex , 2004, Nature Neuroscience.

[53]  L. Chalupa,et al.  The visual neurosciences , 2004 .

[54]  C. Erkelens,et al.  Coordination of hand movements and saccades: evidence for a common and a separate pathway , 2004, Experimental Brain Research.

[55]  Konrad Paul Kording,et al.  Processing of complex stimuli and natural scenes in the visual cortex , 2004, Current Opinion in Neurobiology.

[56]  D. Ballard,et al.  Eye movements in natural behavior , 2005, Trends in Cognitive Sciences.

[57]  L. Itti,et al.  Modeling the influence of task on attention , 2005, Vision Research.

[58]  David J. Field,et al.  How Close Are We to Understanding V1? , 2005, Neural Computation.

[59]  Kathleen A. Turano,et al.  Optic-flow and egocentric-direction strategies in walking: Central vs peripheral visual field , 2005, Vision Research.

[60]  L. Itti.  Author address: , 1999 .

[61]  Jason A. Droll,et al.  Task demands control acquisition and storage of visual information. , 2005, Journal of experimental psychology. Human perception and performance.

[62]  Iain D. Gilchrist,et al.  Visual correlates of fixation selection: effects of scale and time , 2005, Vision Research.

[63]  Ronald A. Rensink,et al.  Change blindness: past, present, and future , 2005, Trends in Cognitive Sciences.

[64]  Roland J. Baddeley,et al.  High frequency edges (but not contrast) predict where we fixate: A Bayesian system identification analysis , 2006, Vision Research.

[65]  P. Subramanian.  Active Vision: The Psychology of Looking and Seeing , 2006 .

[66]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[67]  Garrison W. Cottrell,et al.  A probabilistic model of eye movements in concept formation , 2007, Neurocomputing.

[68]  J. Pelz,et al.  Oculomotor behavior in natural and man-made environments , 2007 .

[69]  T. Miller.  Eye-Hand Coordination , 2008 .

[70]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[71]  Thrasyvoulos N. Pappas,et al.  Proceedings of SPIE: Human Vision and Electronic Imaging XVIII , 2009 .