Maximizing masquerading as matching : Statistical learning and decision-making in choice behavior

1 There has been a long-running debate over whether humans match or maximize when faced with differentially rewarding options under conditions of uncertainty. While maximizing, i.e. consistently choosing the most rewarding option, is theoretically optimal, humans have often been observed to match, i.e. allocating choices stochastically in proportion to the underlying reward rates. Previous models assumed matching behavior to arise from biological limitations or heuristic decision strategies; this, however, would stand in curious contrast to the accumulating evidence that humans have sophisticated machinery for tracking environmental statistics. It begs the questions of why the brain would build sophisticated representations of environmental statistics, only then to adopt a heuristic decision policy that fails to take full advantage of that information. Here, we revisit this debate by presenting data from a novel visual search task, which are shown to favor a particular Bayesian inference and decision-making account over other heuristic and normative models. Specifically, while subjects' first-fixation strategy appears to indicate matching in aggregate data, they actually maximize on a finer, trial-by-trial timescale, based on continuously updated internal beliefs about the spatial distribution of potential target locations. In other words, matching-like stochasticity in human visual search is neither random nor heuristics-based, but due specifically to fluctuating beliefs about stimulus statistics. These results not only shed light on the matching versus maximizing debate, but also more broadly on human decision-making strategies under conditions of uncertainty.

[1]  C. Eriksen,et al.  Visual attention within and around the field of focal attention: A zoom lens model , 1986, Perception & psychophysics.

[2]  M. Hayhoe,et al.  In what ways do eye movements contribute to everyday activities? , 2001, Vision Research.

[3]  Michael D. Lee,et al.  Psychological models of human and optimal performance in bandit problems , 2011, Cognitive Systems Research.

[4]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[5]  J. Deutsch,et al.  Attention: Some theoretical considerations. , 1963 .

[6]  Wilson S. Geisler,et al.  Optimal eye movement strategies in visual search , 2005, Nature.

[7]  Angela J. Yu,et al.  Bayesian Prediction and Evaluation in the Anterior Cingulate Cortex , 2013, The Journal of Neuroscience.

[8]  K. Rayner Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[9]  M. Roesch,et al.  Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. , 2003, Journal of neurophysiology.

[10]  H. Helmholtz The Facts in Perception , 1977 .

[11]  Thomas L. Griffiths,et al.  One and Done? Optimal Decisions From Very Few Samples , 2014, Cogn. Sci..

[12]  Alan Poling,et al.  Human Performance Under Sequences of Fixed-Ratio Schedules: Effects of Ratio Size and Magnitude of Reinforcement , 1988 .

[13]  Peter Dayan,et al.  Inference, Attention, and Decision in a Bayesian Neural Architecture , 2004, NIPS.

[14]  A. L. Yarbus Eye movements and vision Plenum Press , 1967 .

[15]  H. Basford,et al.  Optimal eye movement strategies in visual search , 2005 .

[16]  D. Broadbent Perception and communication , 1958 .

[17]  R. Herrnstein On the law of effect. , 1970, Journal of the experimental analysis of behavior.

[18]  Angela J. Yu Bayesian Models of Attention , 2014 .

[19]  Krista A. Ehinger,et al.  Modelling search for people in 900 scenes: A combined source model of eye guidance , 2009 .

[20]  E. Soetens,et al.  Expectancy or Automatic Facilitation? Separating Sequential Effects in Two-Choice Reaction Time , 1985 .

[21]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[22]  Angela J. Yu,et al.  Dynamics of attentional selection under conflict: toward a rational Bayesian account. , 2009, Journal of experimental psychology. Human perception and performance.

[23]  W. Newsome,et al.  Matching Behavior and the Representation of Value in the Parietal Cortex , 2004, Science.

[24]  J. Henderson Regarding Scenes , 2007 .

[25]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[26]  G Gigerenzer,et al.  Reasoning the fast and frugal way: models of bounded rationality. , 1996, Psychological review.

[27]  Peter Dayan,et al.  ACh, Uncertainty, and Cortical Inference , 2001, NIPS.

[28]  R J HERRNSTEIN,et al.  Relative and absolute strength of response as a function of frequency of reinforcement. , 1961, Journal of the experimental analysis of behavior.

[29]  J. Deutsch Perception and Communication , 1958, Nature.

[30]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[31]  R H S Carpenter,et al.  The time course of stimulus expectation in a saccadic decision task. , 2007, Journal of neurophysiology.

[32]  Jonathan D. Cohen,et al.  Sequential effects: Superstition or rational behavior? , 2008, NIPS.

[33]  Eileen Kowler,et al.  The role of location probability in the programming of saccades: Implications for “center-of-gravity” tendencies , 1989, Vision Research.

[34]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[35]  A. Poling,et al.  Preference in pigeons given a choice between sequences of fixed-ratio schedules: Effects of ratio values and duration of food delivery. , 1984, Journal of the experimental analysis of behavior.

[36]  M. Nowak,et al.  A strategy of win-stay, lose-shift that outperforms tit-for-tat in the Prisoner's Dilemma game , 1993, Nature.

[37]  M. Lee,et al.  A Bayesian analysis of human decision-making on bandit problems , 2009 .

[38]  C S Green,et al.  Alterations in choice behavior by manipulations of world model , 2010, Proceedings of the National Academy of Sciences.

[39]  Robert C. Wilson,et al.  Rational regulation of learning dynamics by pupil–linked arousal systems , 2012, Nature Neuroscience.

[40]  Matt Jones,et al.  Sequential Effects Reflect Parallel Learning of Multiple Environmental Regularities , 2022 .

[41]  T. Zentall,et al.  Win-stay/lose-shift and win-shift/lose-stay learning by pigeons in the absence of overt response mediation , 1997, Behavioural Processes.

[42]  R. Zemel,et al.  Statistical models and sensory attention , 1999 .

[43]  W. Newsome,et al.  Choosing the greater of two goods: neural currencies for valuation and decision making , 2005, Nature Reviews Neuroscience.

[44]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[45]  C. Koch,et al.  Task-demands can immediately reverse the effects of sensory-driven saliency in complex visual stimuli. , 2008, Journal of vision.

[46]  C. Koch Strategies and Models of Selective Attention , 2010 .

[47]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[48]  J. Warren Reversal learning and the formation of learning sets by cats and rhesus monkeys. , 1966, Journal of Comparative and Physiological Psychology.

[49]  David N. Lee,et al.  Where we look when we steer , 1994, Nature.