Dimensionality, information and learning in prefrontal cortex

Learning leads to changes in population patterns of neural activity. In this study we wanted to examine how these changes in patterns of activity affect the dimensionality of neural responses and information about choices. We addressed these questions by carrying out high channel count recordings in dorsal-lateral prefrontal cortex (dlPFC; 768 electrodes) while monkeys performed a two-armed bandit reinforcement learning task. The high channel count recordings allowed us to study population coding while monkeys learned choices between actions or objects. We found that the dimensionality of neural population activity was higher across blocks in which animals learned the values of novel pairs of objects, than across blocks in which they learned the values of actions. The increase in dimensionality with learning in object blocks was related to less shared information across blocks, and therefore patterns of neural activity that were less similar, when compared to learning in action blocks. Furthermore, these differences emerged with learning, and were not a simple function of the choice of a visual image or action. Therefore, learning the values of novel objects increases the dimensionality of neural representations in dlPFC.

[1]  E. Miller,et al.  The Role of Prefrontal Dopamine D1 Receptors in the Neural Mechanisms of Associative Learning , 2012, Neuron.

[2]  H. Sompolinsky,et al.  Compressed sensing, sparsity, and dimensionality in neuronal information processing and data analysis. , 2012, Annual review of neuroscience.

[3]  T. Sejnowski Neural populations revealed , 1988, Nature.

[4]  Alice M Stamatakis,et al.  Excitatory transmission from the amygdala to nucleus accumbens facilitates reward seeking. , 2011, Nature.

[5]  W. Newsome,et al.  Context-dependent computation by recurrent dynamics in prefrontal cortex , 2013, Nature.

[6]  Joel Z. Leibo,et al.  Prefrontal cortex as a meta-reinforcement learning system , 2018, bioRxiv.

[7]  Daeyeol Lee,et al.  Effects of noise correlations on information encoding and decoding. , 2006, Journal of neurophysiology.

[8]  Kenneth D. Harris,et al.  High-dimensional geometry of population responses in visual cortex , 2019, Nat..

[9]  Andrew R. Mitz,et al.  Subcortical Substrates of Explore-Exploit Decisions in Primates , 2019, Neuron.

[10]  James J. DiCarlo,et al.  How Does the Brain Solve Visual Object Recognition? , 2012, Neuron.

[11]  Kenneth D. Miller,et al.  Coupling between One-Dimensional Networks Reconciles Conflicting Dynamics in LIP and Reveals Its Recurrent Circuitry , 2017, Neuron.

[12]  Byron M. Yu,et al.  Stimulus-Driven Population Activity Patterns in Macaque Primary Visual Cortex , 2016, PLoS Comput. Biol..

[13]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[14]  B. Averbeck,et al.  Injection of a Dopamine Type 2 Receptor Antagonist into the Dorsal Striatum Disrupts Choices Driven by Previous Outcomes, But Not Perceptual Inference , 2015, The Journal of Neuroscience.

[15]  Stefano Fusi,et al.  Hebbian Learning in a Random Network Captures Selectivity Properties of the Prefrontal Cortex , 2017, The Journal of Neuroscience.

[16]  Joel L. Davis,et al.  Adaptive Critics and the Basal Ganglia , 1995 .

[17]  Byron M. Yu,et al.  New neural activity patterns emerge with long-term learning , 2019, Proceedings of the National Academy of Sciences.

[18]  S. Floresco,et al.  Preferential Involvement by Nucleus Accumbens Shell in Mediating Probabilistic Learning and Reversal Shifts , 2014, The Journal of Neuroscience.

[19]  Andrew R. Mitz,et al.  High channel count single-unit recordings from nonhuman primate frontal cortex , 2017, Journal of Neuroscience Methods.

[20]  Ilana B. Witten,et al.  Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target , 2016, Nature Neuroscience.

[21]  Ha Hong,et al.  Performance-optimized hierarchical models predict neural responses in higher visual cortex , 2014, Proceedings of the National Academy of Sciences.

[22]  Byron M. Yu,et al.  Neural constraints on learning , 2014, Nature.

[23]  Josiah R. Boivin,et al.  A Causal Link Between Prediction Errors, Dopamine Neurons and Learning , 2013, Nature Neuroscience.

[24]  A. Pouget,et al.  Information-limiting correlations , 2014, Nature Neuroscience.

[25]  Surya Ganguli,et al.  A theory of multineuronal dimensionality, dynamics and measurement , 2017, bioRxiv.

[26]  D. Barraclough,et al.  Prefrontal cortex and decision making in a mixed-strategy game , 2004, Nature Neuroscience.

[27]  B. Averbeck,et al.  Reinforcement learning in artificial and biological systems , 2019, Nature Machine Intelligence.

[28]  Xiao-Jing Wang,et al.  The importance of mixed selectivity in complex cognitive tasks , 2013, Nature.

[29]  S. Nicola,et al.  Basolateral Amygdala Neurons Facilitate Reward-Seeking Behavior by Exciting Nucleus Accumbens Neurons , 2008, Neuron.

[30]  B. Averbeck,et al.  Amygdala Contributions to Stimulus–Reward Encoding in the Macaque Medial and Orbital Frontal Cortex during Learning , 2017, The Journal of Neuroscience.

[31]  R. Dolan,et al.  Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans , 2006, Nature.

[32]  S. Haber,et al.  Reward-Related Cortical Inputs Define a Large Striatal Region in Primates That Interface with Associative Cortical Connections, Providing a Substrate for Incentive-Based Learning , 2006, The Journal of Neuroscience.

[33]  M. Sahani,et al.  Cortical control of arm movements: a dynamical systems perspective. , 2013, Annual review of neuroscience.

[34]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[35]  Brent Doiron,et al.  Scaling Properties of Dimensionality Reduction for Neural Populations and Network Models , 2016, PLoS Comput. Biol..

[36]  Byron M. Yu,et al.  Learning by neural reassociation , 2018, Nature Neuroscience.

[37]  S. Kay Fundamentals of statistical signal processing: estimation theory , 1993 .

[38]  B. Averbeck,et al.  Action Selection and Action Value in Frontal-Striatal Circuits , 2012, Neuron.

[39]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[40]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[41]  Emad N. Eskandar,et al.  A flexible software tool for temporally-precise behavioral control in Matlab , 2008, Journal of Neuroscience Methods.

[42]  Vincent D Costa,et al.  Amygdala and Ventral Striatum Make Distinct Contributions to Reinforcement Learning , 2016, Neuron.

[43]  Apostolos P. Georgopoulos,et al.  Neural activity in prefrontal cortex during copying geometrical shapes , 2003, Experimental Brain Research.

[44]  Joel L. Davis,et al.  A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement , 1994 .

[45]  K. Miller,et al.  One-Dimensional Dynamics of Attention and Decision Making in LIP , 2008, Neuron.

[46]  Byron M. Yu,et al.  Dimensionality reduction for large-scale neural recordings , 2014, Nature Neuroscience.

[47]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[48]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[49]  Naoshige Uchida,et al.  Demixed principal component analysis of neural population data , 2014, eLife.

[50]  D. Durstewitz,et al.  Abrupt Transitions between Prefrontal Neural Ensemble States Accompany Behavioral Transitions during Rule Learning , 2010, Neuron.

[51]  Vaughn L. Hetrick,et al.  Mesolimbic Dopamine Signals the Value of Work , 2015, Nature Neuroscience.

[52]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[53]  E. Miller,et al.  Different time courses of learning-related activity in the prefrontal cortex and striatum , 2005, Nature.

[54]  Anna S. Mitchell,et al.  Critical role for the mediodorsal thalamus in permitting rapid reward-guided updating in stochastic reward environments , 2016, eLife.

[55]  Michael J. Frank,et al.  Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism , 2005, Journal of Cognitive Neuroscience.

[56]  A. Pouget,et al.  Neural correlations, population coding and computation , 2006, Nature Reviews Neuroscience.

[57]  S. Wise,et al.  Learning-dependent neuronal activity in the premotor cortex: activity during the acquisition of conditional motor associations , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[58]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[59]  John P. Cunningham,et al.  Tensor Analysis Reveals Distinct Population Structure that Parallels the Different Computational Roles of Areas M1 and V1 , 2016, PLoS Comput. Biol..

[60]  Matthew P. H. Gardner,et al.  Optogenetic Blockade of Dopamine Transients Prevents Learning Induced by Changes in Reward Features , 2017, Current Biology.

[61]  Matthew T. Kaufman,et al.  Neural population dynamics during reaching , 2012, Nature.

[62]  Bruno B. Averbeck,et al.  Amygdala and ventral striatum population codes implement multiple learning rates for reinforcement learning , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[63]  H. Seo,et al.  Neural basis of reinforcement learning and decision making. , 2012, Annual review of neuroscience.

[64]  Surya Ganguli,et al.  On simplicity and complexity in the brave new world of large-scale neuroscience , 2015, Current Opinion in Neurobiology.

[65]  Greg O. Horne,et al.  Controlling low-level image properties: The SHINE toolbox , 2010, Behavior research methods.

[66]  Daeyeol Lee,et al.  Activity in prefrontal cortex during dynamic selection of action sequences , 2006, Nature Neuroscience.

[67]  Vincent D Costa,et al.  Motivational neural circuits underlying reinforcement learning , 2017, Nature Neuroscience.

[68]  Kathryn M. Rothenhoefer,et al.  Effects of Ventral Striatum Lesions on Stimulus-Based versus Action-Based Reinforcement Learning , 2017, The Journal of Neuroscience.

[69]  John P. Cunningham,et al.  Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity , 2008, NIPS.

[70]  E. Miller,et al.  Neural Activity in the Primate Prefrontal Cortex during Associative Learning , 1998, Neuron.

[71]  T. Bussey,et al.  Effects of selective thalamic and prelimbic cortex lesions on two types of visual discrimination and reversal learning , 2001, The European journal of neuroscience.