Value-guided remapping of sensory circuits by lateral orbitofrontal cortex in reversal learning

Flexible decision-making is crucial for adaptive behaviour. Such behaviour in mammals largely relies on the frontal cortex, and specifically, the orbitofrontal cortex (OFC). How OFC neurons encode decision variables and instruct sensory areas to guide adaptive behaviour is a key open question. Here we developed a reversal learning task for head-fixed mice together with two-photon calcium imaging to monitor the activity of lateral OFC neuronal populations and investigated their dynamic interaction with primary somatosensory cortex (S1). Mice trained on this task learned to discriminate go/no-go tactile stimuli and adapt their behaviour upon changes in stimulus–reward contingencies (‘rule-switch’). Longitudinal imaging at cellular resolution across weeks during all behavioural phases revealed a distinct engagement of S1 and lateral OFC neurons: S1 neural activity reflected task learning-related responses, while neurons in the lateral OFC saliently and transiently responded to the rule-switch. A subset of OFC neurons conveyed a value prediction error signal via feedback projections to S1, as direct anatomical long-range projections were revealed by retrograde tracing combined with whole-brain light-sheet microscopy. Top-down signals implemented an update of sensory representations and functionally reconfigured a small subpopulation of S1 neurons that were differentially modulated by reward-history. Functional remapping of these neurons crucially depended on top-down inputs, as chemogenetic silencing of lateral OFC neurons disrupted reversal learning and impaired plastic changes in these outcome-sensitive S1 neurons. Our results reveal the presence of long-range cortical interactions between cellular ensembles in higher and lower-order brain areas specifically recruited during context-dependent learning and task-switching. Such interactions crucially implement history-dependent reward-value computations and error heuristics, which, in turn, help guide adaptive behaviour.

[1]  James H. Marshel,et al.  Interacting neural ensembles in orbitofrontal cortex for social and feeding behaviour , 2019, Nature.

[2]  W. Schultz Multiple reward signals in the brain , 2000, Nature Reviews Neuroscience.

[3]  M. Bear,et al.  Reward Timing in the Primary Visual Cortex , 2006, Science.

[4]  F. Helmchen,et al.  Behaviour-dependent recruitment of long-range projection neurons in somatosensory cortex , 2013, Nature.

[5]  A. Izquierdo Functional Heterogeneity within Rat Orbitofrontal Cortex in Reward Learning and Decision Making , 2017, The Journal of Neuroscience.

[6]  E. Miller,et al.  The prefontral cortex and cognitive control , 2000, Nature Reviews Neuroscience.

[7]  R. Naik Ramesh,et al.  Intermingled Ensembles in Visual Association Cortex Encode Stimulus Identity or Predicted Outcome , 2018, Neuron.

[8]  E. Murray,et al.  The Orbitofrontal Oracle: Cortical Mechanisms for the Prediction and Evaluation of Specific Behavioral Outcomes , 2014, Neuron.

[9]  Aditya Gilra,et al.  Thalamic regulation of switching between cortical representations enables cognitive flexibility , 2018, Nature Neuroscience.

[10]  T. Robbins,et al.  Dissociable Contributions of the Orbitofrontal and Infralimbic Cortex to Pavlovian Autoshaping and Discrimination Reversal Learning: Further Evidence for the Functional Heterogeneity of the Rodent Frontal Cortex , 2003, The Journal of Neuroscience.

[11]  J. Wallis Orbitofrontal cortex and its contribution to decision-making. , 2007, Annual review of neuroscience.

[12]  E. Rolls The orbitofrontal cortex and reward. , 2000, Cerebral cortex.

[13]  M. Roesch,et al.  Interneurons Are Necessary for Coordinated Activity During Reversal Learning in Orbitofrontal Cortex , 2015, Biological Psychiatry.

[14]  Karl J. Friston,et al.  Canonical Microcircuits for Predictive Coding , 2012, Neuron.

[15]  Marie Carlén,et al.  What constitutes the prefrontal cortex? , 2017, Science.

[16]  M. Scanziani,et al.  How Inhibition Shapes Cortical Activity , 2011, Neuron.

[17]  F. Helmchen,et al.  Pathway-specific reorganization of projection neurons in somatosensory cortex during learning , 2015, Nature Neuroscience.

[18]  J. Downar,et al.  Cortico-Striatal-Thalamic Loop Circuits of the Orbitofrontal Cortex: Promising Therapeutic Targets in Psychiatric Illness , 2017, Front. Syst. Neurosci..

[19]  M. Roesch,et al.  A new perspective on the role of the orbitofrontal cortex in adaptive behaviour , 2009, Nature Reviews Neuroscience.

[20]  Joseph J. Paton,et al.  Distinct Roles for the Amygdala and Orbitofrontal Cortex in Representing the Relative Amount of Expected Reward , 2017, Neuron.

[21]  Nicolas W. Schuck,et al.  Human Orbitofrontal Cortex Represents a Cognitive Map of State Space , 2016, Neuron.

[22]  B. Averbeck,et al.  Reinforcement learning in artificial and biological systems , 2019, Nature Machine Intelligence.

[23]  Jung Hoon Sul,et al.  Distinct Roles of Rodent Orbitofrontal and Medial Prefrontal Cortex in Decision Making , 2010, Neuron.

[24]  E. Murray,et al.  Dissociable Effects of Subtotal Lesions within the Macaque Orbital Prefrontal Cortex on Reward-Guided Behavior , 2011, The Journal of Neuroscience.

[25]  A. Dickinson,et al.  Neuronal coding of prediction errors. , 2000, Annual review of neuroscience.

[26]  Stephanie M. Groman,et al.  Orbitofrontal Circuits Control Multiple Reinforcement-Learning Processes , 2019, Neuron.

[27]  Timothy E. J. Behrens,et al.  Review Frontal Cortex and Reward-guided Learning and Decision-making Figure 1. Frontal Brain Regions in the Macaque Involved in Reward-guided Learning and Decision-making Finer Grained Anatomical Divisions with Frontal Cortical Systems for Reward-guided Behavior , 2022 .

[28]  T. Robbins,et al.  Serotonin Modulates Sensitivity to Reward and Negative Feedback in a Probabilistic Reversal Learning Task in Rats , 2010, Neuropsychopharmacology.

[29]  Mriganka Sur,et al.  Task-dependent representations of stimulus and choice in mouse parietal cortex , 2017, Nature Communications.

[30]  C. Summerfield,et al.  Where Does Value Come From? , 2019, Trends in Cognitive Sciences.

[31]  J. Fuster The Prefrontal Cortex—An Update Time Is of the Essence , 2001, Neuron.