Task-set Selection in Probabilistic Environments: a Model of Task-set Inference

To act effectively in a complicated, uncertain world, people often rely on task-sets (TSs) that define action policies over a range of stimuli. Effectively selecting amongst TSs requires assessing their individual utility given the current world state. However, the world state is, in general, latent, stochastic, and time-varying, making TS selection a difficult inference for the agent. An open question is how observable environmental factors influence an actor's assessment of the world state and thus the selection of TSs. In this work, we designed a novel task in which probabilistic cues predict one of two TSs on a trial-by-trial basis. With this task, we investigate how people integrate multiple sources of probabilistic information in the service of TS selection. We show that when action feedback is unavailable, TS selection can be modeled as “biased Bayesian inference”, such that individuals participants differentially weight immediate cues over TS priors when inferring the latent world state. Additionally, using the model’s trial-bytrial posteriors over TSs, we calculate a measure of decision confidence and show that it inversely relates to reaction times. This work supports the hierarchical organization of decision-making by demonstrating that probabilistic evidence can be integrated in the service of higher-order decisions over TSs, subsequently simplifying lower-order action selection.

[1]  John R. Anderson,et al.  The Adaptive Nature of Human Categorization , 1991 .

[2]  John R. Anderson,et al.  The Adaptive Nature of Human Categorization. , 1991 .

[3]  D. Blei,et al.  Context, learning, and extinction. , 2010, Psychological review.

[4]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[5]  Jadin C. Jackson,et al.  Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. , 2007, Psychological review.

[6]  V. Henmon,et al.  The relation of the time of a judgment to its accuracy. , 1911 .

[7]  M. Shadlen,et al.  Decision Making as a Window on Cognition , 2013, Neuron.

[8]  M. Frank,et al.  Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis. , 2012, Cerebral cortex.

[9]  Anne G E Collins,et al.  Cognitive control over learning: creating, clustering, and generalizing task-set structure. , 2013, Psychological review.

[10]  Mitsuo Kawato,et al.  Multiple Model-Based Reinforcement Learning , 2002, Neural Computation.

[11]  M. D’Esposito,et al.  Frontal Cortex and the Discovery of Abstract Action Rules , 2010, Neuron.

[12]  M. Shadlen,et al.  Response of Neurons in the Lateral Intraparietal Area during a Combined Visual Discrimination Reaction Time Task , 2002, The Journal of Neuroscience.

[13]  Alec Solway,et al.  Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates. , 2012, Psychological review.

[14]  I. Erev,et al.  On adaptation, maximization, and reinforcement learning among cognitive strategies. , 2005, Psychological review.

[15]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[16]  Joshua B. Tenenbaum,et al.  A probabilistic model of cross-categorization , 2011, Cognition.

[17]  J. Tenenbaum,et al.  Theory-based Bayesian models of inductive learning and reasoning , 2006, Trends in Cognitive Sciences.

[18]  E. Koechlin,et al.  Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making , 2012, PLoS biology.

[19]  Y. Niv,et al.  Learning latent structure: carving nature at its joints , 2010, Current Opinion in Neurobiology.