Eye movements as a window to cognitive processes

Eye movement research is a highly active and productive research field. Here we focus on how the embodied nature of eye movements can act as a window to the brain and the mind. In particular, we discuss how conscious perception depends on the trajectory of fixated locations and consequently address how fixation locations are selected. Specifically, we argue that the selection of fixation points during visual exploration can be understood to a large degree based on retinotopically structured models. Yet, these models largely ignore spatiotemporal structure in eye-movement sequences. Explaining spatiotemporal structure in eye-movement trajectories requires an understanding of spatiotemporal properties of the visual sampling process. With this in mind, we discuss the availability of external information to internal inference about causes in the world. We demonstrate that visual foraging is a dynamic process that can be systematically modulated either towards exploration or exploitation. For an analysis at high temporal resolution, we suggest a new method: The renewal density allows the investigation of precise temporal relation of eye movements and other actions like a button press. We conclude with an outlook and propose that eye movement research has reached an appropriate stage and can easily be combined with other research methods to utilize this window to the brain and mind to its fullest.

[1]  T. Weidner The Human Gaze: , 2018, Humanitarianism and Media.

[2]  A. Nuthmann Fixation durations in scene viewing: Modeling the effects of local image features, oculomotor parameters, and task , 2016, Psychonomic bulletin & review.

[3]  R. Venkatesh Babu,et al.  DeepFix: A Fully Convolutional Neural Network for Predicting Human Eye Fixations , 2015, IEEE Transactions on Image Processing.

[4]  Zhaoping Li,et al.  From the optic tectum to the primary visual cortex: migration through evolution of the saliency map for exogenous attentional guidance , 2016, Current Opinion in Neurobiology.

[5]  K. Gegenfurtner The Interaction Between Vision and Eye Movements † , 2016, Perception.

[6]  U. Ansorge,et al.  Action selection as a guide for visual attention , 2016 .

[7]  Qi Zhao,et al.  SALICON: Reducing the Semantic Gap in Saliency Prediction by Adapting Deep Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Martina Poletti,et al.  Control and Functions of Fixational Eye Movements. , 2015, Annual review of vision science.

[9]  Ocular Movements in Reversibility of Perspective , 2015 .

[10]  Radek Ptak,et al.  The role of visual saliency for the allocation of attention: Evidence from spatial neglect and hemianopia , 2015, Neuropsychologia.

[11]  Peter König,et al.  Predictions of Visual Content across Eye Movements and Their Modulation by Inferred Information , 2015, The Journal of Neuroscience.

[12]  P. König,et al.  Effects of contextual information and stimulus ambiguity on overt visual sampling behavior , 2015, Vision Research.

[13]  Gernot Horstmann,et al.  The surprise–attention link: a review , 2015, Annals of the New York Academy of Sciences.

[14]  C. Constantinidis,et al.  Bottom-Up and Top-Down Attention , 2014, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[15]  Oded Berger-Tal,et al.  The Exploration-Exploitation Dilemma: A Multidisciplinary Framework , 2014, PloS one.

[16]  Vinzenz von Tscharner,et al.  Methodological aspects of EEG and body dynamics measurements during motion , 2014, Front. Hum. Neurosci..

[17]  P. König,et al.  Spatial biases in viewing behavior. , 2014, Journal of vision.

[18]  Peter König,et al.  Dissociation between saliency signals and activity in early visual cortex. , 2013, Journal of vision.

[19]  Jose L. Contreras-Vidal,et al.  Simultaneous scalp electroencephalography (EEG), electromyography (EMG), and whole-body segmental inertial recording for multi-modal neural decoding. , 2013, Journal of visualized experiments : JoVE.

[20]  V. Lamme,et al.  Bottom-up and top-down attention are independent. , 2013, Journal of vision.

[21]  A. Clark Whatever next? Predictive brains, situated agents, and the future of cognitive science. , 2013, The Behavioral and brain sciences.

[22]  Peter König,et al.  Saccadic Momentum and Facilitation of Return Saccades Contribute to an Optimal Foraging Strategy , 2013, PLoS Comput. Biol..

[23]  P. König,et al.  Unmasking the contribution of low-level features to the guidance of attention , 2012, Neuropsychologia.

[24]  P. König,et al.  Combining EEG and eye tracking: identification, characterization, and correction of eye movement artifacts in electroencephalographic data , 2012, Front. Hum. Neurosci..

[25]  Janneke F. M. Jehee,et al.  Attention Reverses the Effect of Prediction in Silencing Sensory Signals , 2011, Cerebral cortex.

[26]  Erhardt Barth,et al.  Impact of dynamic bottom-up features and top-down control on the visual exploration of moving real-world scenes in hemispatial neglect , 2012, Neuropsychologia.

[27]  Andreas K. Engel,et al.  The saccadic spike artifact in MEG , 2012, NeuroImage.

[28]  Zhaoping Li,et al.  Neural Activities in V1 Create a Bottom-Up Saliency Map , 2012, Neuron.

[29]  Peter König,et al.  Viewing behavior and the impact of low-level image properties across repeated presentations of complex scenes. , 2011, Journal of vision.

[30]  Peter König,et al.  Measures and Limits of Models of Fixation Selection , 2011, PloS one.

[31]  J. Henderson,et al.  Does Oculomotor Inhibition of Return Influence Fixation Probability during Scene Search? This Strategy Would Require a Mechanism for Keeping Track of Visited Locations , 2022 .

[32]  Peter König,et al.  Overt Visual Attention as a Causal Factor of Perceptual Awareness , 2011, PloS one.

[33]  Eileen Kowler Eye movements: The past 25years , 2011, Vision Research.

[34]  M. Carrasco Visual attention: The past 25 years , 2011, Vision Research.

[35]  E. Knudsen,et al.  Control from below: the role of a midbrain network in spatial attention , 2011, The European journal of neuroscience.

[36]  Christof Koch,et al.  Visual Saliency Computations: Mechanisms, Constraints, and the Effect of Feedback , 2010, The Journal of Neuroscience.

[37]  P. König,et al.  Getting real—sensory processing of natural stimuli , 2010, Current Opinion in Neurobiology.

[38]  Attention and its coupling to action. , 2010, British journal of psychology.

[39]  Peter König,et al.  Influence of Low-Level Stimulus Features, Task Dependent Factors, and Spatial Biases on Overt Visual Attention , 2010, PLoS Comput. Biol..

[40]  Jon Driver,et al.  Integration of Goal- and Stimulus-Related Visual Signals Revealed by Damage to Human Parietal Cortex , 2010, The Journal of Neuroscience.

[41]  J. Henderson,et al.  CRISP: a computational model of fixation durations in scene viewing. , 2010, Psychological review.

[42]  Tim C Kietzmann,et al.  Investigating task-dependent top-down effects on overt visual attention. , 2010, Journal of vision.

[43]  Caspar M. Schwiedrzik,et al.  Stimulus Predictability Reduces Responses in Primary Visual Cortex , 2010, The Journal of Neuroscience.

[44]  R. Ptak,et al.  Looking left with left neglect: The role of spatial attention when active vision selects local image features for fixation , 2009, Cortex.

[45]  P. Glimcher,et al.  Visual processing, learning and feedback in the primate eye movement system , 2009, Trends in Neurosciences.

[46]  M. Hayhoe,et al.  Adaptive Gaze Control in Natural Environments , 2009, The Journal of Neuroscience.

[47]  Peter König,et al.  Eye–Head Coordination during Free Exploration in Human and Cat , 2009, Annals of the New York Academy of Sciences.

[48]  R. Müri,et al.  Visual exploration pattern in hemineglect , 2009, Psychological research.

[49]  Peter König,et al.  Influence of disparity on fixation and saccades in free viewing of natural scenes. , 2009, Journal of vision.

[50]  Rudolf Groner,et al.  The effect of spatial frequency content on parameters of eye movements , 2008, Psychological research.

[51]  Peter König,et al.  What's color got to do with it? The influence of color on visual attention in different categories. , 2008, Journal of vision.

[52]  Peter König,et al.  Salient features in gaze-aligned recordings of human visual input during free exploration of natural environments. , 2008, Journal of vision.

[53]  L. Zhaoping Attention capture by eye of origin singletons even without awareness--a hallmark of a bottom-up saliency map in the primary visual cortex. , 2008, Journal of vision.

[54]  Michael J. Constantino,et al.  Neural repetition suppression reflects fulfilled perceptual expectations , 2008 .

[55]  M. Paré,et al.  Neuronal activity in superior colliculus signals both stimulus identity and saccade goals during visual conjunction search. , 2007, Journal of vision.

[56]  Benjamin W Tatler,et al.  The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. , 2007, Journal of vision.

[57]  P. Wurtz,et al.  Size Matters: Saccades during Scene Perception , 2007, Perception.

[58]  Peter König,et al.  Human eye-head co-ordination in natural exploration , 2007, Network.

[59]  R. Baddeley,et al.  The long and the short of it: Spatial statistics at fixation vary with saccade amplitude and task , 2006, Vision Research.

[60]  Asha Iyer,et al.  Components of bottom-up gaze allocation in natural images , 2005, Vision Research.

[61]  B. Velichkovsky,et al.  Time course of information processing during scene perception: The relationship between saccade amplitude and fixation duration , 2005 .

[62]  D. Ballard,et al.  Eye movements in natural behavior , 2005, Trends in Cognitive Sciences.

[63]  Iain D. Gilchrist,et al.  Visual correlates of fixation selection: effects of scale and time , 2005, Vision Research.

[64]  Rudolf Groner,et al.  Attention and eye movement control: An overview , 2005, European archives of psychiatry and neurological sciences.

[65]  Peter König,et al.  Are switches in perception of the Necker cube related to eye position? , 2004, The European journal of neuroscience.

[66]  Michael L. Mack,et al.  Human Gaze Control in RealWorld Search , 2004, WAPCV.

[67]  Konrad Paul Kording,et al.  Bayesian integration in sensorimotor learning , 2004, Nature.

[68]  J. Gallant,et al.  Goal-Related Activity in V4 during Free Viewing Visual Search Evidence for a Ventral Stream Visual Salience Map , 2003, Neuron.

[69]  S. Treue Visual attention: the where, what, how and why of saliency , 2003, Current Opinion in Neurobiology.

[70]  Antonio Torralba,et al.  Modeling global scene factors in attention. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[71]  Junji Ito,et al.  Perceptual Switching, Eye Movements, and the Bus Paradox , 2003, Perception.

[72]  P. König,et al.  Does luminance‐contrast contribute to a saliency map for overt visual attention? , 2003, The European journal of neuroscience.

[73]  D. Ballard,et al.  What you see is what you need. , 2003, Journal of vision.

[74]  Paul Schrater,et al.  Shape perception reduces activity in human primary visual cortex , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[75]  Edward H. Adelson,et al.  Motion illusions as optimal percepts , 2002, Nature Neuroscience.

[76]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[77]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[78]  M. Hayhoe,et al.  In what ways do eye movements contribute to everyday activities? , 2001, Vision Research.

[79]  J. Pelz,et al.  Oculomotor behavior and perceptual strategies in complex tasks , 2001, Vision Research.

[80]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[81]  Michael F. Land,et al.  From eye movements to actions: how batsmen hit the ball , 2000, Nature Neuroscience.

[82]  Leslie G. Ungerleider,et al.  Mechanisms of visual attention in the human cortex. , 2000, Annual review of neuroscience.

[83]  H. Pashler The Psychology of Attention , 1997 .

[84]  Michael I. Jordan,et al.  An internal model for sensorimotor integration. , 1995, Science.

[85]  David N. Lee,et al.  Where we look when we steer , 1994, Nature.

[86]  S. Petersen,et al.  The pulvinar and visual salience , 1992, Trends in Neurosciences.

[87]  O. Mimura [Eye movements]. , 1992, Nippon Ganka Gakkai zasshi.

[88]  J. Theeuwes Exogenous and endogenous control of attention: The effect of visual onsets and offsets , 1991, Perception & psychophysics.

[89]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[90]  J. Marshall This is psychology? , 1981, Nature.

[91]  Nobuo Kawabata,et al.  Visual fixation points and depth perception , 1978, Vision Research.

[92]  D. Hubel,et al.  Uniformity of monkey striate cortex: A parallel relationship between field size, scatter, and magnification factor , 1974, The Journal of comparative neurology.

[93]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[94]  Eure Sp,et al.  Reversible figures and eye-movements. , 1956 .

[95]  S. Eure,et al.  Reversible figures and eye-movements. , 1956, The American journal of psychology.

[96]  Nancy Millette How People Look at Pictures. By Guy T. Buswell , 1936 .

[97]  W. Gooding,et al.  Psychology and Pedagogy , 1895, The School Review.

[98]  C. Wheatstone XVIII. Contributions to the physiology of vision. —Part the first. On some remarkable, and hitherto unobserved, phenomena of binocular vision , 1962, Philosophical Transactions of the Royal Society of London.

[99]  Charles Wheatstone,et al.  Contributions to the Physiology of Vision. , 1837 .

[100]  L. A. N. Esq.,et al.  LXI. Observations on some remarkable optical phænomena seen in Switzerland; and on an optical phænomenon which occurs on viewing a figure of a crystal or geometrical solid , 1832 .