Subjective optimality in finite sequential decision-making

Many decisions in life are sequential and constrained by a time window. Although mathematically derived optimal solutions exist, it has been reported that humans often deviate from making optimal choices. Here, we used a secretary problem, a classic example of finite sequential decision-making, and investigated the mechanisms underlying individuals’ suboptimal choices. Across three independent experiments, we found that a dynamic programming model comprising subjective value function explains individuals’ deviations from optimality and predicts the choice behaviors under fewer opportunities. We further identified that pupil dilation reflected the levels of decision difficulty and subsequent choices to accept or reject the stimulus at each opportunity. The value sensitivity, a model-based estimate that characterizes each individual’s subjective valuation, correlated with the extent to which individuals’ physiological responses tracked stimuli information. Our results provide model-based and physiological evidence for subjective valuation in finite sequential decision-making, rediscovering human suboptimality in subjectively optimal decision-making processes.

[1]  Ian Krajbich,et al.  Visual fixations and the computation and comparison of value in simple choice , 2010, Nature Neuroscience.

[2]  T. Knapen,et al.  Decision-related pupil dilation reflects upcoming choice and individual bias , 2014, Proceedings of the National Academy of Sciences.

[3]  Robert C. Wilson,et al.  Rational regulation of learning dynamics by pupil–linked arousal systems , 2012, Nature Neuroscience.

[4]  Theodore P. Hill,et al.  Minimax-Optimal Stop Rules and Distributions in Secretary Problems , 1991 .

[5]  P. Freeman The Secretary Problem and its Extensions: A Review , 1983 .

[6]  Thomas S. Ferguson,et al.  Who Solved the Secretary Problem , 1989 .

[7]  T. Ohshima,et al.  Stimulated emission from nitrogen-vacancy centres in diamond , 2016, Nature Communications.

[8]  Joseph T. McGuire,et al.  Decision making and the avoidance of cognitive demand. , 2010, Journal of experimental psychology. General.

[9]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[10]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[11]  Colin Camerer,et al.  A framework for studying the neurobiology of value-based decision making , 2008, Nature Reviews Neuroscience.

[12]  Thomas V. Wiecki,et al.  Eye tracking and pupillometry are indicators of dissociable latent decision processes. , 2014, Journal of experimental psychology. General.

[13]  Michael D. Lee,et al.  The Effect of Goals and Environments on Human Performance in Optimal Stopping Problems , 2018, Decision.

[14]  Vincent D Costa,et al.  Frontal-parietal and limbic-striatal activity underlies information sampling in the best choice problem. , 2015, Cerebral cortex.

[15]  Teresa A. Victor,et al.  Aberrant decision-making and drug addiction—how strong is the evidence? , 2017, Current Opinion in Behavioral Sciences.

[16]  R. Preston McAfee,et al.  Learning When to Stop Searching , 2020, Manag. Sci..

[17]  Nathaniel D. Daw,et al.  Trial-by-trial data analysis using computational models , 2011 .

[18]  Timothy Edward John Behrens,et al.  Value, search, persistence and model updating in anterior cingulate cortex , 2016, Nature Neuroscience.

[19]  S. M. Samuels,et al.  Optimal selection based on relative rank (the “secretary problem”) , 1964 .

[20]  Jan Theeuwes,et al.  Cognitive and Ocular Factors Jointly Determine Pupil Responses under Equiluminance , 2016, PloS one.

[21]  G. Loewenstein,et al.  Anomalies in Intertemporal Choice: Evidence and an Interpretation , 1992 .

[22]  Ryan O. Murphy,et al.  Experimental Studies of Sequential Selection and Assignment with Relative Ranks , 2006 .

[23]  Mark A. Straccia,et al.  Anterior Cingulate Engagement in a Foraging Context Reflects Choice Difficulty, Not Foraging Value , 2014, Nature Neuroscience.

[24]  M. Milinski,et al.  Costs influences sequential mate choice in sticklebacks, Gasterosteus aculeatus , 1992, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[25]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[26]  M. Lee,et al.  The Effect of Feedback and Financial Reward on Human Performance Solving 'Secretary' Problems , 2006 .

[27]  Peter Muris,et al.  To be or not to be...indecisive: Gender differences, correlations with obsessive-compulsive complaints, and behavioural manifestation , 2005 .

[28]  Raymond J. Dolan,et al.  Anterior cingulate activity during error and autonomic response , 2005, NeuroImage.

[29]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[30]  D. Hamilton,et al.  A Categorization Approach to Stereotyping , 2015 .

[31]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[32]  John M. Pearson,et al.  Neuronal basis of sequential foraging decisions in a patchy environment , 2011, Nature Neuroscience.

[33]  Darryl A. Seale,et al.  Sequential Decision Making with Relative Ranks: An Experimental Investigation of the "Secretary Problem"> , 1997 .

[34]  D. Hamilton Cognitive Processes in Stereotyping and Intergroup Behavior , 1981 .

[35]  Anne E. Urai,et al.  Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias , 2017, Nature Communications.

[36]  Christoph W Korn,et al.  How unrealistic optimism is maintained in the face of reality , 2011, Nature Neuroscience.

[37]  Jan Theeuwes,et al.  How pupil responses track value-based decision-making during and after reinforcement learning , 2018, PLoS Comput. Biol..

[38]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[39]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[40]  A. Soltani,et al.  Belief about nicotine selectively modulates value and reward prediction error signals in smokers , 2015, Proceedings of the National Academy of Sciences.

[41]  Geoffrey F. Yeo Interview costs in the secretary problem , 1998 .