Rapid Learning of Odor–Value Association in the Olfactory Striatum

Rodents can successfully learn multiple novel stimulus–response associations after only a few repetitions when the contingencies predict reward. The circuits modified during such reinforcement learning to support decision-making are not known, but the olfactory tubercle (OT) and posterior piriform cortex (pPC) are candidates for decoding reward category from olfactory sensory input and relaying this information to cognitive and motor areas. Through single-cell recordings in behaving male and female C57BL/6 mice, we show here that an explicit representation for reward category emerges in the OT within minutes of learning a novel odor–reward association, whereas the pPC lacks an explicit representation even after weeks of overtraining. The explicit reward category representation in OT is visible in the first sniff (50–100 ms) of an odor on each trial, and precedes the motor action. Together, these results suggest that the coding of stimulus information required for reward prediction does not occur within olfactory cortex, but rather in circuits involving the olfactory striatum. SIGNIFICANCE STATEMENT Rodents are olfactory specialists and can use odors to learn contingencies quickly and well. We have found that mice can readily learn to place multiple odors into rewarded and unrewarded categories. Once they have learned the rule, they can do such categorization in a matter of minutes (<10 trials). We found that neural activity in olfactory cortex largely reflects sensory coding, with very little explicit information about categories. By contrast, neural activity in a brain region in the ventral striatum is rapidly modified in a matter of minutes to reflect reward category. Our experiments set up a paradigm for studying rapid sensorimotor reinforcement in a circuit that is right at the interface of sensory input and reward areas.

[1]  W. Welker Analysis of Sniffing of the Albino Rat 1) , 1964 .

[2]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[3]  L. Heimer,et al.  Ventral striatum and ventral pallidum Components of the motor system? , 1982, Trends in Neurosciences.

[4]  L. Heimer,et al.  Cell configurations in the olfactory tubercle of the rat , 1984, The Journal of comparative neurology.

[5]  D. S. Zahm,et al.  The ventral striatopallidothalamic projection: I. The striatopallidal link originating in the striatal parts of the olfactory tubercle , 1987, The Journal of comparative neurology.

[6]  B. McNaughton,et al.  Tetrodes markedly improve the reliability and yield of multiple single-unit isolation from multi-unit recordings in cat striate cortex , 1995, Journal of Neuroscience Methods.

[7]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[8]  L. Haberly,et al.  New Features of Connectivity in Piriform Cortex Visualized by Intracellular Injection of Pyramidal Cells Suggest that “Primary” Olfactory Cortex Functions Like “Association” Cortex in Other Sensory Systems , 2000, The Journal of Neuroscience.

[9]  W. Schultz Multiple reward signals in the brain , 2000, Nature Reviews Neuroscience.

[10]  B. Slotnick,et al.  Can rats acquire an olfactory learning set? , 2000, Journal of experimental psychology. Animal behavior processes.

[11]  Burton Slotnick,et al.  Animal cognition and the rat olfactory system , 2001, Trends in Cognitive Sciences.

[12]  D. Tolhurst,et al.  Characterizing the sparseness of neural codes , 2001, Network.

[13]  John N. J. Reynolds,et al.  Dopamine-dependent plasticity of corticostriatal synapses , 2002, Neural Networks.

[14]  S. Ikemoto Involvement of the Olfactory Tubercle in Cocaine Reward: Intracranial Self-Administration Studies , 2003, The Journal of Neuroscience.

[15]  Z. Mainen,et al.  Speed and accuracy of olfactory discrimination in the rat , 2003, Nature Neuroscience.

[16]  J. Hollerman,et al.  Changes in behavior-related neuronal activity in the striatum during learning , 2003, Trends in Neurosciences.

[17]  L. Haberly,et al.  Odor‐evoked activity is spatially distributed in piriform cortex , 2003, The Journal of comparative neurology.

[18]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[19]  Andreas T. Schaefer,et al.  Maintaining Accuracy at the Expense of Speed Stimulus Similarity Defines Odor Discrimination Time in Mice , 2004, Neuron.

[20]  E. Miller,et al.  Different time courses of learning-related activity in the prefrontal cortex and striatum , 2005, Nature.

[21]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[22]  Tomaso Poggio,et al.  Fast Readout of Object Identity from Macaque Inferior Temporal Cortex , 2005, Science.

[23]  Yonatan Loewenstein,et al.  Operant matching is a generic outcome of synaptic plasticity based on the covariance between reward and neural activity , 2006, Proceedings of the National Academy of Sciences.

[24]  A. Gelperin,et al.  Speed-Accuracy Tradeoff in Olfaction , 2006, Neuron.

[25]  M. Roesch,et al.  Associative encoding in posterior piriform cortex during odor discrimination and reversal learning. , 2007, Cerebral cortex.

[26]  S. Ikemoto Dopamine reward circuitry: Two projection systems from the ventral midbrain to the nucleus accumbens–olfactory tubercle complex , 2007, Brain Research Reviews.

[27]  Adam Kepecs,et al.  Rapid and precise control of sniffing during olfactory discrimination in rats. , 2007, Journal of neurophysiology.

[28]  D. Wesson,et al.  Sniffing behavior of mice during performance in odor-guided tasks. , 2008, Chemical senses.

[29]  P. Dayan,et al.  Reinforcement learning: The Good, The Bad and The Ugly , 2008, Current Opinion in Neurobiology.

[30]  Dan D. Stettler,et al.  Representations of Odor in the Piriform Cortex , 2009, Neuron.

[31]  Wolfgang Maass,et al.  Reward-Modulated Hebbian Learning of Decision Making , 2010, Neural Computation.

[32]  J. Gottfried Central mechanisms of odour object perception , 2010, Nature Reviews Neuroscience.

[33]  Naoshige Uchida,et al.  Robust Odor Coding via Inhalation-Coupled Transient Activity in the Mammalian Olfactory Bulb , 2010, Neuron.

[34]  Donald A. Wilson,et al.  Cortical Processing of Odor Objects , 2011, Neuron.

[35]  Matthew C Smear,et al.  Precise olfactory responses tile the sniff cycle , 2011, Nature Neuroscience.

[36]  Donald A. Wilson,et al.  Sniffing out the contributions of the olfactory tubercle to the sense of smell: Hedonics, sensory integration, and more? , 2011, Neuroscience & Biobehavioral Reviews.

[37]  Anne E Carpenter,et al.  Neuron-type specific signals for reward and punishment in the ventral tegmental area , 2011, Nature.

[38]  Naoshige Uchida,et al.  Odor Representations in Olfactory Cortex: Distributed Rate Coding and Decorrelated Population Activity , 2012, Neuron.

[39]  Sachie K. Ogawa,et al.  Whole-Brain Mapping of Direct Inputs to Midbrain Dopamine Neurons , 2012, Neuron.

[40]  A. Zador,et al.  Corticostriatal neurones in auditory cortex drive decisions during auditory discrimination , 2013, Nature.

[41]  Jennifer D. Whitesell,et al.  Information for Decision Making and Stimulus Identification is Multiplexed in Sensory Cortex , 2013, Nature Neuroscience.

[42]  Selective corticostriatal plasticity during acquisition of an auditory discrimination task , 2014 .

[43]  Sandeep Robert Datta,et al.  Olfactory maps, circuits and computations , 2014, Current Opinion in Neurobiology.

[44]  V. Murthy,et al.  An olfactory cocktail party: figure-ground segregation of odorants in rodents , 2014, Nature Neuroscience.

[45]  Ha Hong,et al.  Simple Learned Weighted Sums of Inferior Temporal Neuronal Firing Rates Accurately Predict Human Core Object Recognition Performance , 2015, The Journal of Neuroscience.

[46]  Gonzalo H. Otazu,et al.  Cortical Feedback Decorrelates Olfactory Bulb Output in Awake Mice , 2015, Neuron.

[47]  W. Kelsch,et al.  Phasic Dopamine Modifies Sensory-Driven Output of Striatal Neurons through Synaptic Plasticity , 2015, The Journal of Neuroscience.

[48]  D. Wesson,et al.  The Olfactory Tubercle Encodes Odor Valence in Behaving Mice , 2015, The Journal of Neuroscience.

[49]  A. Zador,et al.  Selective corticostriatal plasticity during acquisition of an auditory discrimination task , 2014, Nature.

[50]  D. Wesson,et al.  The Neural Representation of Goal-Directed Actions and Outcomes in the Ventral Striatum's Olfactory Tubercle , 2016, The Journal of Neuroscience.

[51]  Matthias Bethge,et al.  Reading Out Olfactory Receptors: Feedforward Circuits Detect Odors in Mixtures without Demixing , 2016, Neuron.

[52]  Alexander Fleischmann,et al.  Molecular signatures of neural connectivity in the olfactory cortex , 2016, Nature Communications.

[53]  Fuqiang Xu,et al.  Whole-Brain Mapping of the Inputs and Outputs of the Medial Part of the Olfactory Tubercle , 2017, Front. Neural Circuits.

[54]  Sandeep Robert Datta,et al.  Population Coding in an Innately Relevant Olfactory Area , 2017, Neuron.

[55]  M. Yamaguchi Functional Sub-Circuits of the Olfactory System Viewed from the Olfactory Bulb and the Olfactory Tubercle , 2017, Front. Neuroanat..

[56]  Kevin M Franks,et al.  Complementary codes for odor identity and intensity in olfactory cortex , 2017, eLife.

[57]  Thomas Deneux,et al.  Odor identity coding by distributed ensembles of neurons in the mouse olfactory cortex , 2017, eLife.

[58]  Zheng Zhou,et al.  Activation of the dopaminergic pathway from VTA to the medial olfactory tubercle generates odor-preference and reward , 2017, eLife.

[59]  D. Wesson,et al.  A neural system for encoding odor valence which influences approach behaviors , 2019, bioRxiv.