Title : Prefrontal cortex state representations shape human credit assignment 1

People learn adaptively from feedback, but the rate of such learning differs drastically 6 across individuals and contexts. Here we examine whether this variability reflects differences 7 in what is learned. Leveraging a neurocomputational approach that merges fMRI and an iterative 8 reward learning task, we link the specificity of credit assignment—how well people are able to 9 appropriately attribute outcomes to their causes—to the precision of neural codes in the 10 prefrontal cortex (PFC). Participants credit task-relevant cues more precisely in social compared 11 to nonsocial contexts, a process that is mediated by high-fidelity (i.e., distinct and consistent) 12 state representations in the PFC. Specifically, the medial PFC and orbitofrontal cortex work in 13 concert to match the neural codes from feedback to those at choice, and the strength of these 14 common neural codes predict credit assignment precision. Together this work provides a window 15 into how neural representations drive adaptive learning. 16 Significance Statement: Successful learning requires selectively attributing outcomes to their 17 cause—a process known as credit assignment. Little is known about how the brain performs this 18 credit assignment, or how the process might differ across contexts or individuals. Functional 19 neuroimaging analyses reveal that precise credit assignment is linked to high fidelity (i.e., 20 distinct and consistent) neural representations of causal cues in the prefrontal cortex (PFC), 21 which supports increased differentiation between stimuli during learning. Our results reveal why 22 individuals learn differently: differences are not driven by the magnitude of learning signals (i.e., 23 stronger prediction errors) as has been previously claimed, but by differences in the strength of 24 neural representations to which those learning signals are attributed. 25

[1]  Avinash R. Vaidya,et al.  Abstract task representations for inference and control , 2022, Trends in Cognitive Sciences.

[2]  Seongmin A. Park,et al.  Neural mechanisms of credit assignment for inferred relationships in a structured world , 2021, Neuron.

[3]  Arif A. Hamid,et al.  Wave-like dopamine dynamics as a mechanism for spatiotemporal credit assignment , 2021, Cell.

[4]  Seongmin A. Park,et al.  The orbital frontal cortex, task structure, and inference. , 2021, Behavioral neuroscience.

[5]  Geoffrey Hunt References and Notes , 2020, The Buddha's Path of Peace: A Step-by-Step Guide.

[6]  M. Nassar,et al.  Latent motives guide structure learning during adaptive social choice , 2020, Nature Human Behaviour.

[7]  M. Frank,et al.  Anxiety Impedes Adaptive Social Learning Under Uncertainty , 2020, Psychological science.

[8]  Joseph T. McGuire,et al.  Dissociable Forms of Uncertainty-Driven Representational Change Across the Human Brain , 2018, The Journal of Neuroscience.

[9]  Lindsay E. Hunter,et al.  Stimulus generalization as a mechanism for learning to trust , 2018, Proceedings of the National Academy of Sciences.

[10]  E. Eskandar,et al.  Prefrontal Neurons Encode a Solution to the Credit-Assignment Problem , 2017, The Journal of Neuroscience.

[11]  Marijn C. W. Kroes,et al.  Threat Intensity Widens Fear Generalization Gradients , 2017, Behavioral neuroscience.

[12]  Lesley K Fellows,et al.  Necessary Contributions of Human Frontal Lobe Subregions to Reward Learning in a Dynamic, Multidimensional Environment , 2016, The Journal of Neuroscience.

[13]  Nicolas W. Schuck,et al.  Human Orbitofrontal Cortex Represents a Cognitive Map of State Space , 2016, Neuron.

[14]  Titus Brooks Heagins,et al.  The MR2: A multi-racial, mega-resolution database of facial stimuli , 2016, Behavior research methods.

[15]  Jörn Diedrichsen,et al.  Reliability of dissimilarity measures for multi-voxel pattern analysis , 2016, NeuroImage.

[16]  Timothy Edward John Behrens,et al.  Reward-Guided Learning with and without Causal Attribution , 2016, Neuron.

[17]  Joshua W. Brown,et al.  Neural Mechanisms of Credit Assignment in a Multicue Environment , 2016, The Journal of Neuroscience.

[18]  C. Gerfen,et al.  Dopamine D2 receptors gate generalization of conditioned threat responses through mTORC1 signaling in the extended amygdala , 2016, Molecular Psychiatry.

[19]  Y. Niv,et al.  Discovering latent causes in reinforcement learning , 2015, Current Opinion in Behavioral Sciences.

[20]  Matthew F.S. Rushworth,et al.  Contrasting Roles for Orbitofrontal Cortex and Amygdala in Credit Assignment and Learning in Macaques , 2015, Neuron.

[21]  J. Dunsmoor,et al.  Fear Generalization and Anxiety: Behavioral and Neural Mechanisms , 2015, Biological Psychiatry.

[22]  Fabian A. Soto,et al.  Explaining compound generalization in associative and causal learning through rational principles of dimensional generalization. , 2014, Psychological review.

[23]  Timothy O. Laumann,et al.  Methods to detect, characterize, and remove motion artifact in resting state fMRI , 2014, NeuroImage.

[24]  Anne G E Collins,et al.  How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis , 2012, The European journal of neuroscience.

[25]  Luke J. Chang,et al.  Seeing is believing: Trustworthiness as a dynamic belief , 2010, Cognitive Psychology.

[26]  R. Paz,et al.  Negative Valence Widens Generalization of Learning , 2010, The Journal of Neuroscience.

[27]  Timothy Edward John Behrens,et al.  Separable Learning Systems in the Macaque Brain and the Role of Orbitofrontal Cortex in Contingent Learning , 2010, Neuron.

[28]  C. Glymour,et al.  Six problems for causal inference from fMRI , 2010, NeuroImage.

[29]  Russell A Poldrack,et al.  Modeling group fMRI data. , 2007, Social cognitive and affective neuroscience.

[30]  Michael J. Frank,et al.  Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia , 2006, Neural Computation.

[31]  C. Grillon,et al.  Classical fear conditioning in the anxiety disorders: a meta-analysis. , 2005, Behaviour research and therapy.

[32]  Thomas E. Nichols,et al.  Validating cluster size inference: random field and permutation methods , 2003, NeuroImage.

[33]  C. L. Hull Principles of behavior : an introduction to behavior theory , 1943 .

[34]  Anne G E Collins,et al.  Cognitive control over learning: creating, clustering, and generalizing task-set structure. , 2013, Psychological review.

[35]  J. B. Tenenbaum,et al.  How to Grow a Mind: Statistics, Structure, and Abstraction , 2011, Science.

[36]  C. Grillon,et al.  Phasic vs Sustained Fear in Rats and Humans: Role of the Extended Amygdala in Fear vs Anxiety , 2010, Neuropsychopharmacology.

[37]  Richard S. Sutton,et al.  Temporal credit assignment in reinforcement learning , 1984 .