Reinstated episodic context guides sampling-based decisions for reward

How does experience inform decisions? In episodic sampling, decisions are guided by a few episodic memories of past choices. This process can yield choice patterns similar to model-free reinforcement learning; however, samples can vary from trial to trial, causing decisions to vary. Here we show that context retrieved during episodic sampling can cause choice behavior to deviate sharply from the predictions of reinforcement learning. Specifically, we show that, when a given memory is sampled, choices (in the present) are influenced by the properties of other decisions made in the same context as the sampled event. This effect is mediated by fMRI measures of context retrieval on each trial, suggesting a mechanism whereby cues trigger retrieval of context, which then triggers retrieval of other decisions from that context. This result establishes a new avenue by which experience can guide choice and, as such, has broad implications for the study of decisions.

[1]  W. Bevan,et al.  The role of context in episodic memory , 1977 .

[2]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[3]  David L. Sjoquist,et al.  Understanding Regression Analysis , 1986 .

[4]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[5]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[6]  Marc W. Howard,et al.  A distributed representation of temporal context , 2002 .

[7]  Mark W. Woolrich,et al.  Advances in functional and structural MR image analysis and implementation as FSL , 2004, NeuroImage.

[8]  P. Glimcher,et al.  JOURNAL OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR 2005, 84, 555–579 NUMBER 3(NOVEMBER) DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS , 2022 .

[9]  Eric J. Johnson,et al.  The Construction of Preference: Constructing Preferences From Memory , 2006 .

[10]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[11]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[12]  Elke U. Weber,et al.  The Construction of Preference: Constructing Preferences From Memory , 2006 .

[13]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[14]  E. Yechiam,et al.  Loss aversion, diminishing sensitivity, and the effect of experience on repeated decisions† , 2008 .

[15]  B. Bernheim On the Potential of Neuroeconomics: A Critical (But Hopeful) Appraisal , 2008 .

[16]  Sean M. Polyn,et al.  A context maintenance and retrieval model of organizational processes in free recall. , 2009, Psychological review.

[17]  Mariana V. C. Coutinho,et al.  Carving nature at its joints using a knife called concepts , 2010, Behavioral and Brain Sciences.

[18]  Y. Niv,et al.  Learning latent structure: carving nature at its joints , 2010, Current Opinion in Neurobiology.

[19]  Sean M. Polyn,et al.  Human memory reconsolidation can be explained using the temporal context model , 2011, Psychonomic bulletin & review.

[20]  D. Shohamy,et al.  Preference by Association: How Memory Mechanisms in the Hippocampus Bias Decisions , 2012, Science.

[21]  Nathaniel D. Daw,et al.  Cortical and Hippocampal Correlates of Deliberation during Model-Based Decisions for Rewards in Humans , 2013, PLoS Comput. Biol..

[22]  Jeremy R. Manning,et al.  The Role of Context in Episodic Memory , 2013 .

[23]  A. Hupbach,et al.  Neural Context Reinstatement Predicts Memory Misattribution , 2013, The Journal of Neuroscience.

[24]  N. Daw,et al.  Integrating memories to guide decisions , 2015, Current Opinion in Behavioral Sciences.

[25]  M. Shadlen,et al.  Decision Making and Sequential Sampling from Memory , 2016, Neuron.

[26]  Larry Schroeder,et al.  Understanding Regression Analysis: An Introductory Guide , 2016 .

[27]  Mel W. Khaw,et al.  Reminders of past choices bias decisions for reward in humans , 2017, Nature Communications.