Optimality and Some of Its Discontents: Successes and Shortcomings of Existing Models for Binary Decisions

We review how leaky competing accumulators (LCAs) can be used to model decision making in two-alternative, forced-choice tasks, and we show how they reduce to drift diffusion (DD) processes in special cases. As continuum limits of the sequential probability ratio test, DD processes are optimal in producing decisions of specified accuracy in the shortest possible time. Furthermore, the DD model can be used to derive a speed-accuracy trade-off that optimizes reward rate for a restricted class of two alternative forced-choice decision tasks. We review findings that compare human performance with this benchmark, and we reveal both approximations to and deviations from optimality. We then discuss three potential sources of deviations from optimality at the psychological level--avoidance of errors, poor time estimation, and minimization of the cost of control--and review recent theoretical and empirical findings that address these possibilities. We also discuss the role of cognitive control in changing environments and in modulating exploitation and exploration. Finally, we consider physiological factors in which nonlinear dynamics may also contribute to deviations from optimality.

[1]  Catalin V. Buhusi,et al.  What makes us tick? Functional and neural mechanisms of interval timing , 2005, Nature Reviews Neuroscience.

[2]  Angela J. Yu,et al.  Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[3]  R. Ratcliff,et al.  A comparison of macaque behavior and superior colliculus neuronal activity to predictions from models of two-choice decisions. , 2003, Journal of neurophysiology.

[4]  James L. McClelland,et al.  Integration of Sensory and Reward Information during Perceptual Decision-Making in Lateral Intraparietal Cortex (LIP) of the Macaque Monkey , 2010, PloS one.

[5]  Han L J van der Maas,et al.  Optimal decision making in neural inhibition models. , 2012, Psychological review.

[6]  J. Gibbon.  Scalar expectancy theory and Weber's law in animal timing. , 1977 .

[7]  James L. McClelland,et al.  A parallel distributed processing approach to automaticity. , 1992, The American journal of psychology.

[8]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[9]  Donald Laming,et al.  Information theory of choice-reaction times , 1968 .

[10]  M. Stone.  Models for choice-reaction time , 1960 .

[11]  M. Shadlen,et al.  Representation of Confidence Associated with a Decision by Neurons in the Parietal Cortex , 2009, Science.

[12]  Kenneth D. Miller,et al.  Mathematical Equivalence of Two Common Forms of Firing Rate Models of Neural Networks , 2012, Neural Computation.

[13]  P. Rabbitt.  Errors and error correction in choice-response tasks. , 1966, Journal of experimental psychology.

[14]  Philip Holmes,et al.  Simple Neural Networks that Optimize Decisions , 2005, Int. J. Bifurc. Chaos.

[15]  J. Wolfowitz,et al.  Optimum Character of the Sequential Probability Ratio Test , 1948 .

[16]  Xiao-Jing Wang,et al.  A Recurrent Network Mechanism of Time Integration in Perceptual Decisions , 2006, The Journal of Neuroscience.

[17]  Philip Holmes,et al.  Can Monkeys Choose Optimally When Faced with Noisy Stimuli and Unequal Rewards? , 2009, PLoS Comput. Biol..

[18]  Jonathan D. Cohen,et al.  An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. , 2005, Annual review of neuroscience.

[19]  W. Todd Maddox,et al.  On the generality of optimal versus objective classifier feedback effects on decision criterion learning in perceptual categorization , 2003, Memory & cognition.

[20]  K. H. Britten,et al.  Responses of neurons in macaque MT to stochastic motion signals , 1993, Visual Neuroscience.

[21]  Joshua W. Brown,et al.  Learned Predictions of Error Likelihood in the Anterior Cingulate Cortex , 2005, Science.

[22]  Naomi Ehrich Leonard,et al.  Can Post-Error Dynamics Explain Sequential Reaction Time Patterns? , 2012, Front. Psychology.

[23]  M. Botvinick,et al.  Conflict monitoring and cognitive control. , 2001, Psychological review.

[24]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[25]  Jonathan D. Cohen,et al.  The neural basis of error detection: conflict monitoring and the error-related negativity. , 2004, Psychological review.

[26]  Andrew M. Saxe,et al.  Acquisition of decision making criteria: reward rate ultimately beats accuracy , 2011, Attention, perception & psychophysics.

[27]  James L. McClelland,et al.  Dynamic Integration of Reward and Stimulus Information in Perceptual Decision-Making , 2011, PloS one.

[28]  Jonathan D. Cohen,et al.  An exploration-exploitation model based on norepinepherine and dopamine activity , 2005, NIPS.

[29]  Philip Holmes,et al.  Rapid decision threshold modulation by reward rate in a neural network , 2006, Neural Networks.

[30]  C. Eriksen,et al.  Pre- and poststimulus activation of response channels: a psychophysiological analysis. , 1988, Journal of experimental psychology. Human perception and performance.

[31]  E. Wagenmakers,et al.  A diffusion model decomposition of the practice effect , 2009, Psychonomic bulletin & review.

[32]  C. Koch,et al.  Pupil dilation reflects perceptual selection and predicts subsequent stability in perceptual rivalry , 2008, Proceedings of the National Academy of Sciences.

[33]  Jonathan D. Cohen,et al.  The Quarterly Journal of Experimental Psychology Do Humans Produce the Speed–accuracy Trade-off That Maximizes Reward Rate? , 2022 .

[34]  Angela J. Yu,et al.  Dynamics of attentional selection under conflict: toward a rational Bayesian account. , 2009, Journal of experimental psychology. Human perception and performance.

[35]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[36]  G. Logan,et al.  When it helps to be misled: Facilitative effects of increasing the frequency of conflicting stimuli in a Stroop-like task , 1979 .

[37]  J. Gold,et al.  Banburismus and the Brain Decoding the Relationship between Sensory Stimuli, Decisions, and Reward , 2002, Neuron.

[38]  Jonathan D. Cohen,et al.  Reward rate optimization in two-alternative decision making: empirical tests of theoretical predictions. , 2009, Journal of experimental psychology. Human perception and performance.

[39]  J D Cohen,et al.  A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior. , 1990, Science.

[40]  Anders Ledberg,et al.  Neurobiological Models of Two-Choice Decision Making Can Be Reduced to a One-Dimensional Nonlinear Diffusion Equation , 2008, PLoS Comput. Biol..

[41]  Jerome R. Busemeyer,et al.  Criterion Learning in a Deferred Decision-Making Task , 1989 .

[42]  J. Cohen,et al.  The role of locus coeruleus in the regulation of cognitive performance. , 1999, Science.

[43]  Jonathan D. Cohen,et al.  A computational model of anterior cingulate function in speeded response tasks: Effects of frequency, sequence, and conflict , 2002, Cognitive, affective & behavioral neuroscience.

[44]  Sander Nieuwenhuis,et al.  Pupil Diameter Predicts Changes in the Exploration–Exploitation Trade-off: Evidence for the Adaptive Gain Theory , 2011, Journal of Cognitive Neuroscience.

[45]  Jonathan D. Cohen,et al.  Mechanisms underlying dependencies of performance on stimulus history in a two-alternative forced-choice task , 2002, Cognitive, affective & behavioral neuroscience.

[46]  Philip Holmes,et al.  A Neural Network Model of the Eriksen Task: Reduction, Analysis, and Data Fitting , 2008, Neural Computation.

[47]  Philip L. Smith,et al.  Dual diffusion model for single-cell recording data from the superior colliculus in a brightness-discrimination task. , 2007, Journal of neurophysiology.

[48]  Joseph T. McGuire,et al.  Effort discounting in human nucleus accumbens , 2009, Cognitive, affective & behavioral neuroscience.

[49]  Jeffrey D. Schall,et al.  Neural basis of deciding, choosing and acting , 2001, Nature Reviews Neuroscience.

[50]  Jonathan D. Cohen,et al.  The Computational and Neural Basis of Cognitive Control: Charted Territory and New Frontiers , 2014, Cogn. Sci..

[51]  C. Eriksen,et al.  Effects of noise letters upon the identification of a target letter in a nonsearch task , 1974 .

[52]  J. Gibbon,et al.  Scalar expectancy theory and peak-interval timing in humans. , 1998, Journal of experimental psychology. Animal behavior processes.

[53]  R. O’Connell,et al.  Pupillometry and P3 index the locus coeruleus-noradrenergic arousal function in humans. , 2011, Psychophysiology.

[54]  Woods Gordon.  The Acquisition Decision , 1985 .

[55]  Eric Shea-Brown,et al.  Optimization of Decision Making in Multilayer Networks: The Role of Locus Coeruleus , 2008, Neural Computation.

[56]  Philip Holmes,et al.  Dynamical Analysis of Bayesian Inference Models for the Eriksen Task , 2009, Neural Computation.

[57]  R. Ratcliff,et al.  Connectionist and diffusion models of reaction time. , 1999, Psychological review.

[58]  J. Townsend,et al.  Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment. , 1993, Psychological review.

[59]  Mark S. Gilzenrat,et al.  Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function , 2010, Cognitive, affective & behavioral neuroscience.

[60]  Jonathan D. Cohen,et al.  Sequential effects: Superstition or rational behavior? , 2008, NIPS.

[61]  G. Kane.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[62]  M Zacksenhouse,et al.  Robust versus optimal strategies for two-alternative forced choice tasks. , 2010, Journal of mathematical psychology.

[63]  Philip Holmes,et al.  Optimality and Robustness of a Biophysical Decision-Making Model under Norepinephrine Modulation , 2009, The Journal of Neuroscience.

[64]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[65]  Philip Holmes,et al.  Dimension Reduction and Dynamics of a Spiking Neural Network Model for Decision Making under Neuromodulation , 2011, SIAM J. Appl. Dyn. Syst..

[66]  M. Shadlen,et al.  Response of Neurons in the Lateral Intraparietal Area during a Combined Visual Discrimination Reaction Time Task , 2002, The Journal of Neuroscience.

[67]  J. Gold,et al.  Neural computations that underlie decisions about sensory stimuli , 2001, Trends in Cognitive Sciences.

[68]  D. Laming.  Choice reaction performance following an error , 1979 .

[69]  Lesley F. Wright,et al.  Information Gap Decision Theory: Decisions under Severe Uncertainty , 2004 .

[70]  Stephen Grossberg,et al.  Nonlinear neural networks: Principles, mechanisms, and architectures , 1988, Neural Networks.

[71]  Philip L. Smith,et al.  Psychology and neurobiology of simple decisions , 2004, Trends in Neurosciences.

[72]  James L. McClelland,et al.  On the control of automatic processes: a parallel distributed processing account of the Stroop effect. , 1990, Psychological review.

[73]  Paul R. Cohen,et al.  ON THE CONTROL OF AUTOMATIC PROCESSES : A PARALLEL DISTRIBUTED PROCESSING MODEL OF THE STROOP EFFECT Technical Report AIP-40 , 2015 .

[74]  Roger Ratcliff,et al.  Dissociable Perceptual-learning Mechanisms Revealed by Diffusion-model Analysis , 2022 .

[75]  M. Shadlen,et al.  A role for neural integrators in perceptual decision making. , 2003, Cerebral cortex.

[76]  Corey J. Bohil,et al.  Base-rate and payoff effects in multidimensional perceptual categorization. , 1998, Journal of Experimental Psychology. Learning, Memory and Cognition.

[77]  James L. McClelland,et al.  The time course of perceptual choice: the leaky, competing accumulator model. , 2001, Psychological review.

[78]  KongFatt Wong-Lin,et al.  Sequential Effects in Two-Choice Reaction Time Tasks: Decomposition and Synthesis of Mechanisms , 2009, Neural Computation.

[79]  Jonathan D. Cohen,et al.  The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. , 2006, Psychological review.

[80]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[81]  M. Posner,et al.  Attention and cognitive control. , 1975 .

[82]  Timothy D. Hanks,et al.  Elapsed Decision Time Affects the Weighting of Prior Probability in a Perceptual Decision Task , 2011, The Journal of Neuroscience.

[83]  L. Jacoby,et al.  Stroop process dissociations: the relationship between facilitation and interference. , 1994, Journal of experimental psychology. Human perception and performance.

[84]  Francis Tuerlinckx,et al.  Do the Dynamics of Prior Information Depend on Task Context? An Analysis of Optimal Performance and an Empirical Test , 2012, Front. Psychology.

[85]  Alan S. Brown,et al.  Information Processing and Cognition: The Loyola Symposium , 1976 .