When is it time to move to the next map? Optimal foraging in guided visual search

Suppose that you are looking for visual targets in a set of images, each containing an unknown number of targets. How do you perform that search, and how do you decide when to move from the current image to the next? Optimal foraging theory predicts that foragers should leave the current image when the expected value from staying falls below the expected value from leaving. Here, we describe how to apply these models to more complex tasks, like search for objects in natural scenes where people have prior beliefs about the number and locations of targets in each image, and search is guided by target features and scene context. We model these factors in a guided search task and predict the optimal time to quit search. The data come from a satellite image search task. Participants searched for small gas stations in large satellite images. We model quitting times with a Bayesian model that incorporates prior beliefs about the number of targets in each map, average search efficiency (guidance), and actual search history in the image. Clicks deploying local magnification were used as surrogates for deployments of attention and, thus, for time. Leaving times (measured in mouse clicks) were well-predicted by the model. People terminated search when their expected rate of target collection fell to the average rate for the task. Apparently, people follow a rate-optimizing strategy in this task and use both their prior knowledge and search history in the image to decide when to quit searching.

[1]  James T. Townsend,et al.  A note on the identifiability of parallel and serial processes , 1971 .

[2]  D. S. Wooding,et al.  Development of the eye-movement response in the trainee radiologist , 1999, Medical Imaging.

[3]  H. Egeth,et al.  Further evidence for a time-independent shift of the focus of attention , 1991, Perception & psychophysics.

[4]  J. Wolfe Moving towards solutions to some enduring controversies in visual search , 2003, Trends in Cognitive Sciences.

[5]  Joel s. Brown,et al.  Foraging : behavior and ecology , 2007 .

[6]  Jeremy M Wolfe,et al.  When do I quit? The search termination problem in visual search. , 2012, Nebraska Symposium on Motivation. Nebraska Symposium on Motivation.

[7]  William G Hayward,et al.  Visual search. , 2013, Wiley interdisciplinary reviews. Cognitive science.

[8]  John McNamara,et al.  A simple model of information use in the exploitation of patchily distributed food , 1985, Animal Behaviour.

[9]  Evan M. Palmer,et al.  Reaction time distributions constrain models of visual search , 2010, Vision Research.

[10]  Preeti Verghese,et al.  The psychophysics of visual search , 2000, Vision Research.

[11]  M. Posner,et al.  Attention and the detection of signals. , 1980, Journal of experimental psychology.

[12]  Michael N. Jones,et al.  Foraging in Semantic Fields: How We Search Through Memory , 2015, Top. Cogn. Sci..

[13]  Jeremy M. Wolfe,et al.  Guided Search 4.0: Current Progress With a Model of Visual Search , 2007, Integrated Models of Cognitive Systems.

[14]  Jeremy M. Wolfe,et al.  Approaches to Visual Search , 2014 .

[15]  J. McNamara Optimal patch use in a stochastic environment , 1982 .

[16]  H L Kundel,et al.  Visual search patterns and experience with radiological images. , 1972, Radiology.

[17]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[18]  Horace Barlow,et al.  Measuring the brain's statistical work: From absolute thresholds to autocorrelation in cortex , 2009 .

[19]  Marius Usher,et al.  Competitive guided search: meeting the challenge of benchmark RT distributions. , 2013, Journal of vision.

[20]  J. Wolfe,et al.  The role of memory for visual search in scenes , 2015, Annals of the New York Academy of Sciences.

[21]  Ola Olsson,et al.  Bayes' theorem and its applications in animal behaviour , 2006 .

[22]  Fan Wu,et al.  Robust object-based encoding in visual working memory. , 2013, Journal of vision.

[23]  Jeremy M Wolfe,et al.  When is it time to move to the next raspberry bush? Foraging rules in human visual search. , 2013, Journal of vision.

[24]  Richard F. Green,et al.  Bayesian birds: A simple example of Oaten's stochastic model of optimal foraging , 1980 .

[25]  J. Kruschke Bayesian estimation supersedes the t test. , 2013, Journal of experimental psychology. General.

[26]  J. Palmer Attention in Visual Search: Distinguishing Four Causes of a Set-Size Effect , 1995 .

[27]  David R. Anderson,et al.  AIC model selection and multimodel inference in behavioral ecology: some background, observations, and comparisons , 2011, Behavioral Ecology and Sociobiology.

[28]  R. Rosenholtz,et al.  A summary statistic representation in peripheral vision explains visual search. , 2009, Journal of vision.

[29]  BELA JULESZ,et al.  Rapid discrimination of visual patterns , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[30]  Edward Vul,et al.  A Bayesian Optimal Foraging Model of Human Visual Search , 2012, Psychological science.

[31]  Naomi M. Kenner,et al.  Low target prevalence is a stubborn source of errors in visual search tasks. , 2007, Journal of experimental psychology. General.

[32]  J. Wolfe,et al.  Guided Search 2.0 A revised model of visual search , 1994, Psychonomic bulletin & review.

[33]  H. Egeth,et al.  Searching for conjunctively defined targets. , 1984, Journal of experimental psychology. Human perception and performance.

[34]  Howard E. Egeth,et al.  Parallel processing of multielement displays , 1972 .

[35]  Krista A. Ehinger,et al.  Modelling search for people in 900 scenes: A combined source model of eye guidance , 2009 .

[36]  J. Wolfe,et al.  What attributes guide the deployment of visual attention and how do they do it? , 2004, Nature Reviews Neuroscience.

[37]  Georgia Tourassi,et al.  Receiver Operating Characteristic Analysis: Basic Concepts and Practical Applications , 2018, The Handbook of Medical Image Perception and Techniques.

[38]  R. Rosenholtz,et al.  A summary-statistic representation in peripheral vision explains visual crowding. , 2009, Journal of vision.

[39]  E. Charnov Optimal foraging, the marginal value theorem. , 1976, Theoretical population biology.

[40]  Susan L. Franzel,et al.  Guided search: an alternative to the feature integration model for visual search. , 1989, Journal of experimental psychology. Human perception and performance.

[41]  A. Oaten,et al.  Optimal foraging in patches: a case for stochasticity. , 1977, Theoretical population biology.

[42]  Ashley M. Sherman,et al.  Visual search for arbitrary objects in real scenes , 2011, Attention, perception & psychophysics.

[43]  F. Ohl,et al.  Fallacies in behavioural interpretation of auditory cortex plasticity , 2004, Nature Reviews Neuroscience.

[44]  Miguel P Eckstein,et al.  Visual search: a retrospective. , 2011, Journal of vision.

[45]  Gavriel Salvendy,et al.  Visual search-based design and evaluation of screen magnifiers for older and visually impaired users , 2009, Int. J. Hum. Comput. Stud..

[46]  Peter Pirolli,et al.  Information Foraging , 2009, Encyclopedia of Database Systems.

[47]  Nancy B. Carlisle,et al.  Where do we store the memory representations that guide attention? , 2013, Journal of vision.

[48]  Jeremy M. Wolfe,et al.  Just Say No: How Are Visual Searches Terminated When There Is No Target Present? , 1996, Cognitive Psychology.

[49]  J. Wolfe,et al.  What Can 1 Million Trials Tell Us About Visual Search? , 1998 .

[50]  J. Krebs,et al.  Hunting by expectation or optimal foraging: A study of patch use by chickadees , 1974 .

[51]  Frank E. Ritter,et al.  The Rise of Cognitive Architectures , 2007, Integrated Models of Cognitive Systems.

[52]  Richard F. Green,et al.  Stopping Rules for Optimal Foragers , 1984, The American Naturalist.

[53]  Denis Cousineau,et al.  Termination of a visual search with large display size effects. , 2004, Spatial vision.

[54]  J. Duncan,et al.  Visual search and stimulus similarity. , 1989, Psychological review.

[55]  Gregory J. Zelinsky,et al.  Exploring set size effects in scenes: Identifying the objects of search , 2008 .