MinireviewModeling Task Control of Eye Movements

In natural behavior, visual information is actively sampled from the environment by a sequence of gaze changes. The timing and choice of gaze targets, and the accompanying attentional shifts, are intimately linked with ongoing behavior. Nonetheless, modeling of the deployment of these fixations has been very difficult because they depend on characterizing the underlying task structure. Recently, advances in eye tracking during natural vision, together with the development of probabilistic modeling techniques, have provided insight into how the cognitive agenda might be included in the specification of fixations. These techniques take advantage of the decomposition of complex behaviors into modular components. A particular subset of these models casts the role of fixation as that of providing task-relevant information that is rewarding to the agent, with fixation being selected on the basis of expected reward and uncertainty about environmental state. We review this work here and describe how specific examples can reveal general principles in gaze control.

[1]  Ali Borji,et al.  What/Where to Look Next? Modeling Top-Down Visual Attention in Complex Interactive Environments , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[2]  Mary Hayhoe,et al.  Predicting human visuomotor behaviour in a driving task , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[3]  D. Ballard,et al.  The role of uncertainty and reward on eye movements in a virtual driving task. , 2012, Journal of vision.

[4]  O. Hikosaka,et al.  Robust Representation of Stable Object Values in the Oculomotor Basal Ganglia , 2012, The Journal of Neuroscience.

[5]  J. Gottlieb.  Attention, Learning, and the Value of Information , 2012, Neuron.

[6]  H. Seo,et al.  Neural basis of reinforcement learning and decision making. , 2012, Annual review of neuroscience.

[7]  Alexander C. Schütz,et al.  Dynamic integration of information about salience and value for saccadic eye movements , 2012, Proceedings of the National Academy of Sciences.

[8]  D. Ballard,et al.  Eye guidance in natural vision: reinterpreting salience. , 2011, Journal of vision.

[9]  Dana H. Ballard,et al.  Credit Assignment in Multiple Goal Embodied Visuomotor Behavior , 2010, Front. Psychology.

[10]  M. Goldberg,et al.  Attention, intention, and priority in the parietal lobe. , 2010, Annual review of neuroscience.

[11]  Pietro Perona,et al.  Optimal reward harvesting in complex perceptual environments , 2010, Proceedings of the National Academy of Sciences.

[12]  P. Glimcher,et al.  Visual processing, learning and feedback in the primate eye movement system , 2009, Trends in Neurosciences.

[13]  B. Tatler,et al.  Looking and Acting: Vision and eye movements in natural behaviour , 2009 .

[14]  Matthew H Tong,et al.  SUN: Top-down saliency using natural statistics , 2009, Visual cognition.

[15]  Mary M Hayhoe,et al.  Trade-offs between gaze and working memory use. , 2007, Journal of experimental psychology. Human perception and performance.

[16]  George J. Andersen,et al.  Optical Information for Car Following: The Driving by Visual Angle (DVA) Model , 2007, Hum. Factors.

[17]  H. Seo,et al.  Dynamic signals related to choices and outcomes in the dorsolateral prefrontal cortex. , 2007, Cerebral cortex.

[18]  Dana H. Ballard,et al.  Modeling embodied visual behaviors , 2007, TAP.

[19]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[20]  J. Schall,et al.  Executive control of countermanding saccades by the supplementary eye field , 2006, Nature Neuroscience.

[21]  Jason A. Droll,et al.  Task demands control acquisition and storage of visual information. , 2005, Journal of experimental psychology. Human perception and performance.

[22]  W. Newsome,et al.  Choosing the greater of two goods: neural currencies for valuation and decision making , 2005, Nature Reviews Neuroscience.

[23]  D. Ballard,et al.  Eye movements in natural behavior , 2005, Trends in Cognitive Sciences.

[24]  Rob Gray,et al.  A Two-Point Visual Control Model of Steering , 2004, Perception.

[25]  M. Hayhoe,et al.  In what ways do eye movements contribute to everyday activities? , 2001, Vision Research.

[26]  W. Schultz.  Multiple reward signals in the brain , 2000, Nature Reviews Neuroscience.

[27]  R. Walker,et al.  A model of saccade generation based on parallel processing and competitive inhibition , 1999, Behavioral and Brain Sciences.

[28]  Michael L. Platt,et al.  Neural correlates of decision variables in parietal cortex , 1999, Nature.

[29]  N. P. Bichot,et al.  Saccade target selection in macaque during feature and conjunction visual search , 1999, Visual Neuroscience.

[30]  M. Land,et al.  The Roles of Vision and Eye Movements in the Control of Activities of Daily Living , 1998, Perception.

[31]  M Corbetta,et al.  Frontoparietal cortical networks for directing attention and the eye to visual locations: identical, independent, or overlapping neural systems? , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[32]  H. Pashler.  The Psychology of Attention , 1997 .

[33]  Zoubin Ghahramani,et al.  Learning Dynamic Bayesian Networks , 1997, Summer School on Neural Networks.

[34]  B. Dosher,et al.  The role of attention in the programming of saccades , 1995, Vision Research.

[35]  David N. Lee,et al.  Where we look when we steer , 1994, Nature.

[36]  Ali Borji,et al.  Computational Modeling of Top-down Visual Attention in Interactive Environments , 2011, BMVC.

[37]  Dana H. Ballard,et al.  Recognizing Behavior in Hand-Eye Coordination Patterns , 2009, Int. J. Humanoid Robotics.

[38]  Dario D. Salvucci,et al.  Threaded cognition: an integrated theory of concurrent multitasking. , 2008, Psychological review.

[39]  Colin Camerer,et al.  Neuroeconomics: decision making and the brain , 2008 .

[40]  Antonio Torralba,et al.  Building the gist of a scene: the role of global image features in recognition. , 2006, Progress in brain research.

[41]  D. Ballard,et al.  What you see is what you need. , 2003, Journal of vision.

[42]  Mary M Hayhoe,et al.  Visual memory and motor planning in a natural task. , 2003, Journal of vision.

[43]  Eileen Kowler.  Eye movements and their role in visual and cognitive processes. , 1990, Reviews of oculomotor research.

[44]  S. Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[45]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[46]  R. C. Langford.  How People Look at Pictures, A Study of the Psychology of Perception in Art. , 1936 .