A Suboptimal Model Explained the Spatiotemporal Control of Human Eye Movements in Visual Search

Efficient eye movement planning in both space and time is vital for visual perception. Previous studies showed that human eye movements were consistent with an optimal searcher during visual search; however, we found that such a model could not fully explain the spatial control of eye movements and it completely ignored the temporal control aspect. Here, we first measured the temporal course of human visibility map, and then measured eye movement dynamics during a visual search task. We further built a continuous-time eye movement model that determined both fixation duration and location. When the participants’ saccade amplitude bias and memory capacity limitations were introduced, this model could describe multiple aspects of human visual search behavior much better than the optimal searcher with fixed fixation duration. Interestingly, this model predicted that the human visual memory capacity is approximately 8 fixations. The results showed that human performed suboptimal in visual search, and implied that human may use a mixed control strategy to select fixation location. This strategy may have adaptive advantages considering the limitations of the human brain and statistics of the natural environment.

[1]  Reinhold Kliegl,et al.  No Evidence for a Saccadic Range Effect for Visually Guided and Memory-Guided Saccades in Simple Saccade-Targeting Tasks , 2016, PloS one.

[2]  D. Munoz,et al.  Age-related performance of human subjects on saccadic eye movement tasks , 1998, Experimental Brain Research.

[3]  Björn N.S. Vlaskamp,et al.  Saccadic search: On the duration of a fixation , 2007 .

[4]  Scott D. Brown,et al.  Diffusion Decision Model: Current Issues and History , 2016, Trends in Cognitive Sciences.

[5]  R. J. Beers The Sources of Variability in Saccadic Eye Movements , 2007 .

[6]  Bruce Bridgeman,et al.  Failure to detect displacement of the visual world during saccadic eye movements , 1975, Vision Research.

[7]  Morten H. Christiansen,et al.  A computational model , 2014 .

[8]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[9]  H. Stanley,et al.  Optimizing the success of random searches , 1999, Nature.

[10]  Matthew Heath,et al.  Stimulus-driven saccades are characterized by an invariant undershooting bias: no evidence for a range effect , 2013, Experimental Brain Research.

[11]  Jeremy M. Wolfe,et al.  Guided Search 3.0 , 1997 .

[12]  H. Collewijn,et al.  Binocular co‐ordination of human horizontal saccadic eye movements. , 1988, The Journal of physiology.

[13]  Victor A. F. Lamme,et al.  Are There Multiple Visual Short-Term Memory Stores? , 2008, PloS one.

[14]  Justin M. Ales,et al.  The steady-state visual evoked potential in vision research: A review. , 2015, Journal of vision.

[15]  Ralf Engbert,et al.  ICAT: a computational model for the adaptive control of fixation durations , 2014, Psychonomic bulletin & review.

[16]  Frederick A. A. Kingdom,et al.  Signal Detection Measures , 2016 .

[17]  R A Abrams,et al.  Speed and accuracy of saccadic eye movements: characteristics of impulse variability in the oculomotor system. , 1989, Journal of experimental psychology. Human perception and performance.

[18]  Constantin A. Rothkopf,et al.  Multi-step planning of eye movements in visual search , 2019, Scientific Reports.

[19]  C. McDermott Discrimination , 2009, Inclusive Equality.

[20]  Miguel P Eckstein,et al.  The time course of visual information accrual guiding eye movement decisions. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[21]  Michael D. Lee,et al.  Time-varying boundaries for diffusion models of decision making and response time , 2014, Front. Psychol..

[22]  R. CONRAD Decay Theory of Immediate Memory , 1957, Nature.

[23]  Amelia R Hunt,et al.  Human search for a target on a textured background is consistent with a stochastic model. , 2016, Journal of vision.

[24]  Anthony J. Hornof,et al.  A Computational Model of “Active Vision” for Visual Search in Human–Computer Interaction , 2011, Hum. Comput. Interact..

[25]  P. Lennie,et al.  Information Conveyed by Onset Transients in Responses of Striate Cortical Neurons , 2001, The Journal of Neuroscience.

[26]  Hooge PII: S0042-6989(98)00213-2 , 1998 .

[27]  Li Zhaoping,et al.  A new framework for understanding vision from the perspective of the primary visual cortex , 2019, Current Opinion in Neurobiology.

[28]  Wilson S. Geisler,et al.  Simple summation rule for optimal fixation selection in visual search , 2009, Vision Research.

[29]  J. Schall,et al.  Neural Control of Voluntary Movement Initiation , 1996, Science.

[30]  G. Zelinsky A theory of eye movements during target acquisition. , 2008, Psychological review.

[31]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[32]  C. Hesse,et al.  Grasping remembered objects: Exponential decay of the visual memory , 2010, Vision Research.

[33]  Xavier Daura,et al.  Understanding the Molecular Determinants Driving the Immunological Specificity of the Protective Pilus 2a Backbone Protein of Group B Streptococcus , 2013, PLoS Comput. Biol..

[34]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[35]  Frans A. J. Verstraten,et al.  Saccades Toward the Target Are Planned as Sequences Rather Than as Single Steps , 2013, Psychological science.

[36]  Jean Bullier,et al.  The Timing of Information Transfer in the Visual System , 1997 .

[37]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[38]  K. Rayner,et al.  Eye Movement Control during Reading: Evidence for Direct Control , 1981, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[39]  Marcus Nyström,et al.  An adaptive algorithm for fixation, saccade, and glissade detection in eyetracking data , 2010, Behavior research methods.

[40]  Kazunori O’Hashi,et al.  Mechanisms for shaping receptive field in monkey area TE. , 2017, Journal of neurophysiology.

[41]  H. Collewijn,et al.  Differences in accuracy of human saccades between stationary and jumping targets , 1989, Vision Research.

[42]  Nelson Cowan,et al.  Decay Theory of Immediate Memory: From Brown (1958) to Today (2014) , 2016, Quarterly journal of experimental psychology.

[43]  Eileen Kowler,et al.  Eye movements during visual search: the costs of choosing the optimal path , 2001, Vision Research.

[44]  M. Shadlen,et al.  Response of Neurons in the Lateral Intraparietal Area during a Combined Visual Discrimination Reaction Time Task , 2002, The Journal of Neuroscience.

[45]  Jiri Najemnik,et al.  Eye movement statistics in humans are consistent with an optimal search strategy. , 2008, Journal of vision.

[46]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[47]  Peter König,et al.  Saccadic Momentum and Facilitation of Return Saccades Contribute to an Optimal Foraging Strategy , 2013, PLoS Comput. Biol..

[48]  M. Gutmann,et al.  Fundamentals and Recent Developments in Approximate Bayesian Computation , 2016, Systematic biology.

[49]  C. J. Erkelens,et al.  Control of fixation duration in a simple search task , 1996, Perception & psychophysics.

[50]  Frank J. Bruggeman,et al.  The number of active metabolic pathways is bounded by the number of cellular constraints at maximal metabolic rates , 2018, bioRxiv.

[51]  B. Tatler,et al.  The prominence of behavioural biases in eye guidance , 2009 .

[52]  Elise G. Rowe,et al.  Large Capacity of Conscious Access for Incidental Memories in Natural Scenes , 2016, Psychological science.

[53]  J. Rieskamp,et al.  Deciding When to Decide: Time-Variant Sequential Sampling Models Explain the Emergence of Value-Based Decisions in the Human Brain , 2012, The Journal of Neuroscience.

[54]  Christopher A. Dickinson,et al.  Memory for the search path: Evidence for a high-capacity representation of search history , 2007, Vision Research.

[55]  Preeti Verghese,et al.  Efficient saccade planning requires time and clear choices , 2015, Vision Research.

[56]  Antje Nuthmann,et al.  Asymmetrical control of fixation durations in scene viewing , 2014, Vision Research.

[57]  James R. Brockmole,et al.  LATEST: A Model of Saccadic Decisions in Space and Time , 2017, Psychological review.

[58]  A. T. Welford,et al.  Evidence of a Single-Channel Decision Mechanism Limiting Performance in a Serial Reaction Task* , 1959 .

[59]  Gregory J Zelinsky,et al.  TAM: Explaining off-object fixations and central fixation tendencies as effects of population averaging during search , 2012, Visual cognition.

[60]  Tom Foulsham,et al.  Turning the world around: Patterns in saccade direction vary with picture orientation , 2008, Vision Research.

[61]  Jun Kawahara When do additional distractors reduce and increase the attentional blink , 2010 .

[62]  A. Hollingworth,et al.  Visuospatial Working Memory as a Fundamental Component of the Eye Movement System , 2018, Current directions in psychological science.

[63]  Paul Cisek,et al.  Decision making by urgency gating: theory and experimental support. , 2012, Journal of neurophysiology.

[64]  M. Carrasco,et al.  Rapid Simultaneous Enhancement of Visual Sensitivity and Perceived Contrast during Saccade Preparation , 2012, The Journal of Neuroscience.

[65]  John H. R. Maunsell,et al.  The effect of frontal eye field and superior colliculus lesions on saccadic latencies in the rhesus monkey. , 1987, Journal of neurophysiology.

[66]  Ryan V. Ringer,et al.  The spatiotemporal dynamics of scene gist recognition. , 2014, Journal of experimental psychology. Human perception and performance.

[67]  H. Collewijn,et al.  Binocular co‐ordination of human vertical saccadic eye movements. , 1988, The Journal of physiology.

[68]  Melissa R. Beck,et al.  Visual search is guided by prospective and retrospective memory , 2007, Perception & psychophysics.

[69]  V. Lamme,et al.  Fragile visual short-term memory is an object-based and location-specific store , 2013, Psychonomic Bulletin & Review.

[70]  A. Pouget,et al.  The Cost of Accumulating Evidence in Perceptual Decision Making , 2012, The Journal of Neuroscience.

[71]  J. Kaas,et al.  Extrastriate Cortex in Primates , 1997, Cerebral Cortex.

[72]  Björn N. S. Vlaskamp,et al.  Crowding degrades saccadic search performance , 2005, Vision Research.

[73]  Janet Scheel,et al.  Spatiotemporal Dynamics , 2017, Encyclopedia of GIS.

[74]  D. E. Irwin,et al.  How Much Memory Does Oculomotor Search Have? , 2003, Psychological science.

[75]  J. Henderson,et al.  CRISP: a computational model of fixation durations in scene viewing. , 2010, Psychological review.

[76]  Laurence T. Maloney,et al.  Human Visual Search Does Not Maximize the Post-Saccadic Probability of Identifying Targets , 2012, PLoS Comput. Biol..

[77]  Heiko H. Schütt,et al.  Likelihood-Based Parameter Estimation and Comparison of Dynamical Cognitive Models , 2016, Psychological review.

[78]  Zhi Liu,et al.  Saccadic model of eye movements for free-viewing condition , 2015, Vision Research.

[79]  Wilson S. Geisler,et al.  Optimal eye movement strategies in visual search , 2005, Nature.

[80]  Antonio Torralba,et al.  Statistics of natural image categories , 2003, Network.

[81]  A. Jacobs,et al.  Coregistration of eye movements and EEG in natural reading: analyses and review. , 2011, Journal of experimental psychology. General.

[82]  R. Gregory The Most Expensive Painting in the World , 2007, Perception.

[83]  B. Fischer,et al.  Human express saccades: effects of randomization and daily practice , 2004, Experimental Brain Research.

[84]  Axel Meyer,et al.  Asymmetric paralog evolution between the “cryptic” gene Bmp16 and its well-studied sister genes Bmp2 and Bmp4 , 2019, Scientific Reports.

[85]  Reinhold Kliegl,et al.  SWIFT: a dynamical model of saccade generation during reading. , 2005, Psychological review.

[86]  Françoise Vitu,et al.  About the global effect and the critical role of retinal eccentricity: Implications for eye movements in reading , 2008 .

[87]  R. Baloh,et al.  Quantitative measurement of saccade amplitude, duration, and velocity , 1975, Neurology.

[88]  A. Hunt,et al.  Human visual search behaviour is far from ideal , 2017, Proceedings of the Royal Society B: Biological Sciences.

[89]  Roger Ratcliff,et al.  Reinforcement-Based Decision Making in Corticostriatal Circuits: Mutual Constraints by Neurocomputational and Diffusion Models , 2012, Neural Computation.

[90]  Melissa R. Beck,et al.  Memory for where, but not what, is used during visual search. , 2006, Journal of experimental psychology. Human perception and performance.

[91]  F. Kingdom,et al.  Applying the Model-Comparison Approach to Test Specific Research Hypotheses in Psychophysical Research Using the Palamedes Toolbox , 2018, Front. Psychol..

[92]  C. Bruce,et al.  Primate frontal eye fields. II. Physiological and anatomical correlates of electrically evoked eye movements. , 1985, Journal of neurophysiology.

[93]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.