Behavioral theories and the neurophysiology of reward.

! Abstract The functions of rewards are based primarily on their effects on behavior and are less directly governed by the physics and chemistry of input events as in sensory systems. Therefore, the investigation of neural mechanisms underlying reward functions requires behavioral theories that can conceptualize the different effects of rewards on behavior. The scientific investigation of behavioral processes by animal learning theory and economic utility theory has produced a theoretical framework that can help to elucidate the neural correlates for reward functions in learning, goal-directed approach behavior, and decision making under uncertainty. Individual neurons can be studied in the reward systems of the brain, including dopamine neurons, orbitofrontal cortex, and striatum. The neural activity can be related to basic theoretical terms of reward and uncertainty, such as contiguity, contingency, prediction error, magnitude, probability, expected value, and variance.

[1]  S. S. Stevens On the psychophysical law. , 1957, Psychological review.

[2]  A. Harper,et al.  Selection of a solution containing histidine by rats fed a histidine-imbalanced diet. , 1970, Journal of comparative and physiological psychology.

[3]  R. Rescorla A theory of pavlovian conditioning: The effectiveness of reinforcement and non-reinforcement , 1972 .

[4]  H. Markowitsch,et al.  Reward related neurons in cat association cortex , 1976, Brain Research.

[5]  N. Mackintosh,et al.  Surprise and the attenuation of blocking. , 1976 .

[6]  J. Rajkowski,et al.  Tonically discharging putamen neurons exhibit set-dependent responses. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[7]  W. Schultz,et al.  Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. , 1987, Journal of neurophysiology.

[8]  Chi-Fu Huang,et al.  Foundations for financial economics , 1988 .

[9]  H Nishijo,et al.  Single neuron responses in amygdala of alert monkey during complex sensory stimulation with affective significance , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[10]  T. R. Scott,et al.  The responsiveness of neurones in the frontal opercular gustatory cortex of the macaque monkey is independent of hunger. , 1988, The Journal of physiology.

[11]  E. Rolls,et al.  Hunger Modulates the Responses to Gustatory Stimuli of Single Neurons in the Caudolateral Orbitofrontal Cortex of the Macaque Monkey , 1989, The European journal of neuroscience.

[12]  O. Hikosaka,et al.  Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. , 1989, Journal of neurophysiology.

[13]  Masataka Watanabe,et al.  The appropriateness of behavioral responses coded in post-trial activity of primate prefrontal units , 1989, Neuroscience Letters.

[14]  W. Schultz,et al.  Responses of monkey dopamine neurons during learning of behavioral reactions. , 1992, Journal of neurophysiology.

[15]  W. Schultz,et al.  Neuronal activity in monkey ventral striatum related to the expectation of reward , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[16]  A. Mikami,et al.  Activity of single neurons in the monkey amygdala during performance of a visual discrimination task. , 1992, Journal of neurophysiology.

[17]  W. Schultz,et al.  Importance of unpredictability for reward responses in primate dopamine neurons. , 1994, Journal of neurophysiology.

[18]  A. Graybiel,et al.  Responses of tonically active neurons in the primate's striatum undergo systematic changes during behavioral sensorimotor conditioning , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[19]  W. Schultz,et al.  Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli , 1996, Nature.

[20]  E. Rolls,et al.  Hunger and satiety modify the responses of olfactory and visual neurons in the primate orbitofrontal cortex. , 1996, Journal of neurophysiology.

[21]  B. Richmond,et al.  Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards. , 1996, Journal of neurophysiology.

[22]  D. Gietzen,et al.  Temporal-spatial pattern of c-fos expression in the rat brain in response to indispensable amino acid deficiency. I. The initial recognition phase. , 1996, Brain research. Molecular brain research.

[23]  Masataka Watanabe Reward expectancy in primate prefrental neurons , 1996, Nature.

[24]  E. Rolls,et al.  Orbitofrontal cortex neurons: role in olfactory and visual association learning. , 1996, Journal of neurophysiology.

[25]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[26]  S C Rao,et al.  Integration of what and where in the primate prefrontal cortex. , 1997, Science.

[27]  Q. Rogers,et al.  Small changes in essential amino acid concentrations alter diet selection in amino acid-deficient rats. , 1997, The Journal of nutrition.

[28]  B. Richmond,et al.  Neuronal Signals in the Monkey Ventral Striatum Related to Progress through a Predictable Series of Trials , 1998, The Journal of Neuroscience.

[29]  J. Hollerman,et al.  Influence of reward expectation on behavior-related neuronal activity in primate striatum. , 1998, Journal of neurophysiology.

[30]  G. Schoenbaum,et al.  Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning , 1998, Nature Neuroscience.

[31]  J. Hollerman,et al.  Dopamine neurons report an error in the temporal prediction of reward during learning , 1998, Nature Neuroscience.

[32]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[33]  S. Mizumori,et al.  Characteristics of basolateral amygdala neuronal firing on a spatial memory task involving differential reward. , 1998, Behavioral neuroscience.

[34]  O. Hikosaka,et al.  Expectation of reward modulates cognitive signals in the basal ganglia , 1998, Nature Neuroscience.

[35]  J. Hollerman,et al.  Modifications of reward expectation-related neuronal activity during learning in primate striatum. , 1998, Journal of neurophysiology.

[36]  Michael L. Platt,et al.  Neural correlates of decision variables in parietal cortex , 1999, Nature.

[37]  W. Schultz,et al.  Relative reward preference in primate orbitofrontal cortex , 1999, Nature.

[38]  Sabrina Ravel,et al.  Tonically active neurons in the monkey striatum do not preferentially respond to appetitive stimuli , 1999, Experimental Brain Research.

[39]  A. Dickinson,et al.  Neuronal coding of prediction errors. , 2000, Annual review of neuroscience.

[40]  A. Tversky,et al.  Choices, Values, and Frames , 2000 .

[41]  J Schlag,et al.  Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. , 2000, Journal of neurophysiology.

[42]  W. Schultz,et al.  Reward-related neuronal activity during go-nogo task performance in primate orbitofrontal cortex. , 2000, Journal of neurophysiology.

[43]  W. Schultz,et al.  Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. , 2000, Journal of neurophysiology.

[44]  K. Hikosaka,et al.  Delay activity of orbital and lateral prefrontal neurons of the monkey varying with different rewards. , 2000, Cerebral cortex.

[45]  B. Richmond,et al.  Response differences in monkey TE and perirhinal cortex: stimulus association related to reward schedules. , 2000, Journal of neurophysiology.

[46]  W. Schultz,et al.  Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. , 2001, Journal of neurophysiology.

[47]  W. Schultz,et al.  Dopamine responses comply with basic assumptions of formal learning theory , 2001, Nature.

[48]  S. Mizumori,et al.  Neurons in rat medial prefrontal cortex show anticipatory rate changes to predictable differential rewards in a spatial memory task , 2001, Behavioural Brain Research.

[49]  O. Hikosaka,et al.  Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. , 2002, Journal of neurophysiology.

[50]  K. Hikosaka,et al.  Coding and Monitoring of Motivational Context in the Primate Prefrontal Cortex , 2002, The Journal of Neuroscience.

[51]  W. Schultz,et al.  Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm , 2003, The Journal of Neuroscience.

[52]  E. Miller,et al.  Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task , 2003, The European journal of neuroscience.

[53]  Wolfram Schultz,et al.  Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. , 2003, Journal of neurophysiology.

[54]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[55]  Keiji Tanaka,et al.  Neuronal Correlates of Goal-Based Motor Selection in the Prefrontal Cortex , 2003, Science.

[56]  J. C. Crowley,et al.  Saccade Reward Signals in Posterior Cingulate Cortex , 2003, Neuron.

[57]  Tatsuo K Sato,et al.  Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons , 2003, The Journal of Neuroscience.

[58]  J. Bolam,et al.  Uniform Inhibition of Dopamine Neurons in the Ventral Tegmental Area by Aversive Stimuli , 2004, Science.

[59]  O. Hikosaka,et al.  Dopamine Neurons Can Represent Context-Dependent Prediction Error , 2004, Neuron.

[60]  W. Newsome,et al.  Matching Behavior and the Representation of Value in the Parietal Cortex , 2004, Science.

[61]  A. Gelperin,et al.  Post-ingestive food-aversion learning to amino acid deficient diets by the terrestrial slugLimax maximus , 1986, Journal of Comparative Physiology A.

[62]  W. Schultz,et al.  Responses to reward in monkey dorsal and ventral striatum , 2004, Experimental Brain Research.

[63]  J. Maunsell Neuronal representations of cognitive state: reward or attention? , 2004, Trends in Cognitive Sciences.

[64]  E. Vaadia,et al.  Independent Coding of Movement Direction and Reward Prediction by Single Pallidal Neurons , 2004, The Journal of Neuroscience.

[65]  E. Vaadia,et al.  Coincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons , 2004, Neuron.

[66]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[67]  M. Roitman,et al.  Nucleus Accumbens Neurons Are Innately Tuned for Rewarding and Aversive Taste Stimuli, Encode Their Predictors, and Are Linked to Motor Output , 2005, Neuron.

[68]  E. Thorndike Animal Intelligence; Experimental Studies , 2009 .

[69]  K. Campbell,et al.  A neural correlate of response bias in monkey caudate nucleus , 2022 .