Optimal planning of eye movements

The capability of directing gaze to relevant parts in the environment is crucial for our survival. Computational models based on ideal-observer theory have provided quantitative accounts of human gaze selection in a range of visual search tasks. According to these models, gaze is directed to the position in a visual scene, at which uncertainty about task relevant properties will be reduced maximally with the next look. However, in tasks going beyond a single action, delayed rewards can play a crucial role thereby necessitating planning. Here we investigate whether humans are capable of planning more than the next single eye movement. We found evidence that our subjects’ behavior was better explained by an ideal planner compared to the ideal observer. In particular, the location of the first fixation differed depending on the stimulus and the time available for the search. Overall, our results are the first evidence that our visual system is capable of planning.

[1]  Pietro Perona,et al.  Optimal reward harvesting in complex perceptual environments , 2010, Proceedings of the National Academy of Sciences.

[2]  Peter Dayan,et al.  Interplay of approximate planning strategies , 2015, Proceedings of the National Academy of Sciences.

[3]  Quoc C. Vuong,et al.  Influence of encoding context on face recognition , 2010 .

[4]  A. Yuille,et al.  Object perception as Bayesian inference. , 2004, Annual review of psychology.

[5]  Adam Binch,et al.  Perception as Bayesian Inference , 2014 .

[6]  P. Berkes,et al.  Statistically Optimal Perception and Learning: from Behavior to Neural Representations , 2022 .

[7]  Daniel M. Wolpert,et al.  Making smooth moves , 2022 .

[8]  Jitendra Malik,et al.  An Information Maximization Model of Eye Movements , 2004, NIPS.

[9]  W. Geisler.  Ideal Observer Analysis , 2002 .

[10]  R. J. Beers.  The Sources of Variability in Saccadic Eye Movements , 2007 .

[11]  M. Hayhoe,et al.  In what ways do eye movements contribute to everyday activities? , 2001, Vision Research.

[12]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[13]  J. Gottlieb.  Attention, Learning, and the Value of Information , 2012, Neuron.

[14]  D. Ballard,et al.  Eye movements in natural behavior , 2005, Trends in Cognitive Sciences.

[15]  Scott Cheng-Hsin Yang,et al.  Theoretical perspectives on active sensing , 2016, Current Opinion in Behavioral Sciences.

[16]  Frans A. J. Verstraten,et al.  Saccades Toward the Target Are Planned as Sequences Rather Than as Single Steps , 2013, Psychological science.

[17]  Yiannis Aloimonos,et al.  Vision and action , 1995, Image Vis. Comput..

[18]  R. Bellman.  A Markovian Decision Process , 1957 .

[19]  Wilson S. Geisler,et al.  Real-time foveated multiresolution system for low-bandwidth video communication , 1998, Electronic Imaging.

[20]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[21]  Manuel Lopes,et al.  Parietal neurons encode expected gains in instrumental information , 2017, Proceedings of the National Academy of Sciences.

[22]  Adam N. Sanborn,et al.  The Sampling Brain , 2017, Trends in Cognitive Sciences.

[23]  Konrad Paul Kording,et al.  Bayesian integration in sensorimotor learning , 2004, Nature.

[24]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[25]  Scott Cheng-Hsin Yang,et al.  Active sensing in the categorization of visual patterns , 2016, eLife.

[26]  Carolina M. Zingale,et al.  Planning sequences of saccades , 1987, Vision Research.

[27]  W. Geisler,et al.  Contributions of ideal observer theory to vision research , 2011, Vision Research.

[28]  Johan Kwisthout,et al.  Bridging the gap between theory and practice of approximate Bayesian inference , 2013, Cognitive Systems Research.

[29]  N. Chater,et al.  Précis of Bayesian Rationality: The Probabilistic Approach to Human Reasoning , 2009, Behavioral and Brain Sciences.

[30]  M. Wiener,et al.  Animal eyes. , 1957, The American orthoptic journal.

[31]  Wilson S. Geisler,et al.  Optimal eye movement strategies in visual search , 2005, Nature.

[32]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[33]  Samuel J. Gershman,et al.  Computational rationality: A converging paradigm for intelligence in brains, minds, and machines , 2015, Science.

[34]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[35]  P. Subramanian.  Active Vision: The Psychology of Looking and Seeing , 2006 .

[36]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[37]  Alexander C. Schütz,et al.  Dynamic integration of information about salience and value for saccadic eye movements , 2012, Proceedings of the National Academy of Sciences.

[38]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.

[39]  R. J. van Beers,et al.  The Sources of Variability in Saccadic Eye Movements , 2007, The Journal of Neuroscience.

[40]  Constantin A Rothkopf,et al.  Learning rational temporal eye movement strategies , 2016, Proceedings of the National Academy of Sciences.

[41]  J. Tenenbaum,et al.  Theory-based Bayesian models of inductive learning and reasoning , 2006, Trends in Cognitive Sciences.

[42]  Preeti Verghese,et al.  Where to look next? Eye movements reduce local uncertainty. , 2007, Journal of vision.

[43]  Matthew F. Peterson,et al.  Looking just below the eyes is optimal across face recognition tasks , 2012, Proceedings of the National Academy of Sciences.

[44]  Matthew Heath,et al.  Stimulus-driven saccades are characterized by an invariant undershooting bias: no evidence for a range effect , 2013, Experimental Brain Research.

[45]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[46]  R. Baloh,et al.  Quantitative measurement of saccade amplitude, duration, and velocity , 1975, Neurology.

[47]  C. Harris,et al.  Does saccadic undershoot minimize saccadic flight-time? A Monte-Carlo study , 1995, Vision Research.

[48]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[49]  Heiner Deubel,et al.  Properties of attentional selection during the preparation of sequential saccades , 2007, Experimental Brain Research.