The Role of Learning in the Operation of Motivational Systems

Acquired behavior is motivated by two forms of incentive learning. Pavlovian incentive learning reflects the acquisition of motivational properties by conditioned stimuli (CSs) through their association with appetitive and aversive reinforcers. Although the influence of appetitive CSs is modulated by primary motivational states, they exert a general motivation influence on appetitive behavior. By contrast, aversive CSs inhibit appetitive behavior. The second process, instrumental incentive learning, determines the incentive value assigned to outcomes of goal-directed, instrumental action. This incentive value, and its control by primary motivational states, has to be learned through experience of the hedonic reactions elicited by the outcome. The two incentive learning processes function in parallel to motivate instrumental behavior. Keywords: action outcome; appetite-aversive interactions; behavioral chain; blocking; cathexis theory; conditioned stimulus; incentive; motivational control; pavlov; primary drives; stimulus-response

[1]  B. Skinner,et al.  Some quantitative properties of anxiety , 1941 .

[2]  L. Crespi.  Quantitative variation of incentive and performance in the white rat. , 1942 .

[3]  W. Estes.  Discriminative conditioning. I. A discriminative property of conditioned anticipation. , 1943 .

[4]  B. Skinner,et al.  Principles of Behavior , 1944 .

[5]  E. Tolman.  There is more than one kind of learning. , 1949, Psychological review.

[6]  E. Tolman.  The nature and functioning of wants. , 1949, Psychological review.

[7]  C. L. Hull,et al.  A Behavior System , 1954 .

[8]  W. B. Webb.  Drive Stimuli as Cues , 1955 .

[9]  R. Lubow,et al.  Latent inhibition: the effect of nonreinforced pre-exposure to the conditional stimulus. , 1959, Journal of comparative and physiological psychology.

[10]  J. H. Curtis,et al.  Learning Theory and Behavior , 1960 .

[11]  S. Pliskoff,et al.  Water-deprivation-produced sign reversal of a conditioned reinforcer based upon dry food. , 1960, Journal of the experimental analysis of behavior.

[12]  N. Miller,et al.  EFFECT OF STRENGTH OF DRIVE DETERMINED BY A NEW TECHNIQUE FOR APPETITIVE CLASSICAL CONDITIONING OF RATS. , 1965, Journal of comparative and physiological psychology.

[13]  R. Rescorla,et al.  Two-process learning theory: Relationships between Pavlovian conditioning and instrumental learning. , 1967, Psychological review.

[14]  R. Bolles.  Theory of Motivation , 1967 .

[15]  J. Konorski.  Integrative activity of the brain : an interdisciplinary approach , 1967 .

[16]  D. H. Vandercar,et al.  Interstimulus interval functions in different response systems during classical discrimination conditioning of rabbits , 1967 .

[17]  E. E. Krieckhaus,et al.  Acquisition of sodium by rats: interaction of innate mechanisms and latent learning. , 1968, Journal of comparative and physiological psychology.

[18]  V. Lolordo.  Positive conditioned reinforcement from aversive situations. , 1969, Psychological bulletin.

[19]  E. E. Krieckhaus.  "Innate recognition" aids rats in sodium regulation. , 1970, Journal of comparative and physiological psychology.

[20]  Isidore Gormezano,et al.  Effects of water deprivation on classical appetitive conditioning of the rabbit's jaw movement response , 1970 .

[21]  J. S. Myer,et al.  Control of food-motivated instrumental behavior in water-deprived rats by prior water and saline drinking , 1970 .

[22]  S. Mollenauer.  Shifts in deprivation level: Different effects depending on amount of preshift training ☆ , 1971 .

[23]  M. Cabanac.  Physiological Role of Pleasure , 1971, Science.

[24]  J J Franchina,et al.  Reward magnitude shift effects in rats with hippocampal lesions. , 1971, Journal of comparative and physiological psychology.

[25]  I. Gormezano,et al.  Conditioning the rabbit's (Oryctolagus cuniculus) jaw-movement response: US magnitude effects on URs, CRs, and pseudo-CRs. , 1972, Journal of comparative and physiological psychology.

[26]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[27]  D. McFarland,et al.  Hydration releases inhibition of feeding produced by intracranial angiotensin. , 1973, Physiology & behavior.

[28]  D. Bindra.  A motivational view of learning, performance, and behavior modification. , 1974, Psychological review.

[29]  Helen B. Daly,et al.  Reinforcing Properties of Escape From Frustration Aroused In Various Learning Situations , 1974, Psychology of Learning and Motivation.

[30]  A. Wikler,et al.  The First Drink: Psychobiological Aspects of Craving , 1974 .

[31]  R. Rescorla,et al.  The effect of change in US quality on the blocking effect , 1974, The Pavlovian journal of biological science.

[32]  M. G. King,et al.  Heart rate and muscle tension correlates of conditioned suppression in humans. , 1974, Journal of experimental psychology.

[33]  M. J. Scavio.  Classical-classical transfer: effects of prior aversive conditioning upon appetitive conditioning in rabbits (Oryctolagus cuniculus). , 1974, Journal of comparative and physiological psychology.

[34]  A. Dickinson.  Appetitive-aversion interactions: Facilitation of aversive conditioning by prior appetitive training in the rat , 1976 .

[35]  J. Pearce,et al.  Inhibitory interactions between appetitive and aversive stimuli. , 1977 .

[36]  G. Peterson,et al.  Signal Location and Positive Versus Negative Conditioned Suppression in the Rat. , 1977 .

[37]  H. Grill,et al.  The taste reactivity test. I. Mimetic responses to gustatory stimuli in neurologically normal rats , 1978, Brain Research.

[38]  O. K. Fudim,et al.  Sensory preconditioning of flavors with a formalin-produced sodium need. , 1978, Journal of experimental psychology. Animal behavior processes.

[39]  K. Spence.  Behavior Theory and Conditioning , 1978 .

[40]  D. Bindra.  How adaptive behavior is produced: a perceptual-motivational alternative to response reinforcements , 1978, Behavioral and Brain Sciences.

[41]  P. Holland,et al.  Differential effects of omission contingencies on various components of Pavlovian appetitive conditioned responding in rats. , 1979, Journal of experimental psychology. Animal behavior processes.

[42]  R. Hendersen,et al.  Avoidance of heat by rats: Effects of thermal context on rapidity of extinction , 1979 .

[43]  I. Gormezano.  3 – Pavlovian Mechanisms of Goal-Directed Behavior1 , 1980 .

[44]  Richard F. Thompson,et al.  Neural mechanisms of goal-directed behavior and learning , 1980 .

[45]  I. Gormezano,et al.  Classical-classical transfer: Effects of prior appetitive conditioning upon aversive conditioning in rabbits , 1980 .

[46]  Christopher D. Adams,et al.  Instrumental Responding following Reinforcer Devaluation , 1981 .

[47]  D. Booth,et al.  Starch content of ordinary foods associatively conditions human appetite and satiation, indexed by intake and eating pleasantness of starch-paired flavours. , 1982, Appetite.

[48]  Christopher D. Adams.  Variations in the Sensitivity of Instrumental Responding to Reinforcer Devaluation , 1982 .

[49]  A. Dickinson,et al.  Counterconditioning of Appetitive and Defensive CRs in Rabbits , 1982, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[50]  H. Fowler,et al.  Blocking and enhancement of fear conditioning by appetitive CSs , 1983 .

[51]  P. Lovibond.  Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus. , 1983 .

[52]  H. Grill,et al.  Quality of acquired responses to tastes by Rattus norvegicus depends on type of associated discomfort. , 1983, Journal of comparative psychology.

[53]  K. Berridge,et al.  Sodium depletion enhances salt palatability in rats. , 1984, Behavioral neuroscience.

[54]  A. Dickinson.  Actions and habits: the development of behavioural autonomy , 1985 .

[55]  R. Rescorla,et al.  Postconditioning devaluation of a reinforcer affects instrumental responding. , 1985 .

[56]  A. Dickinson.  Re-Examination of the Role of the Instrumental Contingency in the Sodium-Appetite Irrelevant Incentive Effect , 1986, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[57]  R. Rescorla,et al.  Associative Structures In Instrumental Learning , 1986 .

[58]  B. Everitt,et al.  Studies of instrumental behavior with sexual reinforcement in male rats (Rattus norvegicus): II. Effects of preoptic area lesions, castration, and testosterone. , 1987, Journal of comparative psychology.

[59]  J. Pearce,et al.  Pavlovian analysis of interactions between hunger and thirst. , 1987, Journal of experimental psychology. Animal behavior processes.

[60]  T. Davidson.  Learning about deprivation intensity stimuli. , 1987, Behavioral neuroscience.

[61]  A. Dickinson,et al.  Pavlovian Processes in the Motivational Control of Instrumental Performance , 1987 .

[62]  A. Dickinson,et al.  Motivational Control of Instrumental Performance: The Role of Prior Experience of The Reinforcer , 1988 .

[63]  J. Pearce,et al.  Effect of changing the unconditioned stimulus on appetitive blocking. , 1988, Journal of experimental psychology. Animal behavior processes.

[64]  R. Rescorla,et al.  Associations between the discriminative stimulus and the reinforcer in instrumental learning. , 1988 .

[65]  A. Dickinson,et al.  Incentive Learning and the Motivational Control of Instrumental Performance , 1989 .

[66]  A. Dickinson.  Expectancy theory in animal conditioning. , 1989 .

[67]  K. Berridge,et al.  Palatability Shift of a Salt-Associated Incentive during Sodium Depletion , 1989, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[68]  B. Balleine,et al.  Incentive learning and the motivational control of instrumental performance by thirst , 1989 .

[69]  State-dependent fear extinction with two benzodiazepine tranquilizers. , 1990 .

[70]  B. Balleine,et al.  Motivational Control of Instrumental Performance following a Shift from Thirst to Hunger , 1990, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[71]  M. Cabanac,et al.  Postingestive alliesthesia: The rat tells the same story , 1990, Physiology & Behavior.

[72]  M. Cabanac,et al.  Facial consummatory responses in rats support the ponderostat hypothesis , 1991, Physiology & Behavior.

[73]  B. Balleine,et al.  Instrumental Performance following Reinforcer Devaluation Depends upon Incentive Learning , 1991 .

[74]  Kent C. Berridge,et al.  Modulation of taste affect by hunger, caloric satiety, and sensory-specific satiety in the rat , 1991, Appetite.

[75]  A. R. Wagner,et al.  Modulation of a conditioned eyeblink response by a putative emotive stimulus conditioned with hindleg shock. , 1991, Journal of experimental psychology. Animal behavior processes.

[76]  M. Cabanac.  Pleasure: the common currency. , 1992, Journal of theoretical biology.

[77]  R. Rescorla.  Depression of an Instrumental Response by a Single Devaluation of its Outcome , 1992, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[78]  B. Balleine,et al.  Incentive Learning Following Reinforcer Devaluation is not Conditional upon the Motivational State during Re-Exposure , 1992, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[79]  L. Jarrard,et al.  Potency of food deprivation intensity cues as discriminative stimuli. , 1992, Journal of experimental psychology. Animal behavior processes.

[80]  B. Balleine.  Instrumental performance following a shift in primary motivation depends on incentive learning. , 1992, Journal of experimental psychology. Animal behavior processes.

[81]  B. Balleine,et al.  Signalling and Incentive Processes in Instrumental Reinforcer Devaluation , 1992, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[82]  K. Berridge,et al.  The neural basis of drug craving: An incentive-sensitization theory of addiction , 1993, Brain Research Reviews.

[83]  Z. Vickers,et al.  Effect of Flavor and Macronutrient Composition of Food Servings on Liking, Hunger and Subsequent Intake , 1993, Appetite.

[84]  M. Bouton.  Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. , 1993, Psychological bulletin.

[85]  T. Davidson.  The nature and function of interoceptive signals to feed: toward integration of physiological and learning perspectives. , 1993, Psychological review.

[86]  K. Berridge,et al.  Pleasantness of a Sweet Taste during Hunger and Satiety: Effects of Gender and "Sweet Tooth" , 1993, Appetite.

[87]  B. Balleine,et al.  Motivational control of goal-directed action , 1994 .

[88]  M. Bouton.  Context, Ambiguity, and Classical Conditioning , 1994 .

[89]  B. Balleine,et al.  Role of cholecystokinin in the motivational control of instrumental action in rats. , 1994, Behavioral neuroscience.

[90]  R. Rescorla.  A note on Depression of Instrumental Responding after one Trial of Outcome Devaluation , 1994, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[91]  B. Balleine,et al.  Benzodiazepine-induced outcome revaluation and the motivational control of instrumental action in rats. , 1994, Behavioral neuroscience.

[92]  R. Colwill,et al.  Encoding of the unconditioned stimulus in Pavlovian conditioning , 1994 .

[93]  B. Balleine,et al.  Cholecystokinin attenuates incentive learning in rats. , 1995, Behavioral neuroscience.

[94]  B. Balleine,et al.  Instrumental Outcome Devaluation is Attenuated by the Anti-emetic Ondansetron , 1995, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[95]  R. Boakes,et al.  Motivational control after extended instrumental training , 1995 .

[96]  K. Berridge,et al.  Benzodiazepines, appetite, and taste palatability , 1995, Neuroscience & Biobehavioral Reviews.

[97]  B. Balleine,et al.  Motivational control of heterogeneous instrumental chains. , 1995 .

[98]  S. Benoit,et al.  The learned function of food-deprivation cues: A role for conditioned modulation , 1996 .

[99]  B. Balleine,et al.  Role of primary motivation in stimulus preexposure effects. , 1996, Journal of experimental psychology. Animal behavior processes.

[100]  M. Hetherington,et al.  Sensory-specific satiety and its importance in meal termination , 1996, Neuroscience & Biobehavioral Reviews.

[101]  B. Rolls,et al.  Sensory-specific satiety: Theoretical frameworks and central characteristics. , 1996 .

[102]  K. Berridge.  Food reward: Brain substrates of wanting and liking , 1996, Neuroscience & Biobehavioral Reviews.

[103]  R. Colwill,et al.  Direct effects on instrumental performance of outcome revaluation by drive shifts , 1996 .

[104]  A new taste reactivity analysis of the integration of taste and physiological state information. , 1996, The American journal of physiology.

[105]  K. Berridge,et al.  Pimozide Does Not Shift Palatability: Separation of Anhedonia from Sensorimotor Suppression by Taste Reactivity , 1997, Pharmacology Biochemistry and Behavior.

[106]  S. Benoit,et al.  Encoding and selective activation of "metabolic memories" in the rat. , 1997, Behavioral neuroscience.

[107]  T. Davidson.  Hunger cues as modulatory stimuli. , 1998 .

[108]  B. Balleine,et al.  The role of incentive learning in instrumental outcome revaluation by sensory-specific satiety , 1998 .

[109]  R. Beninger,et al.  Dopamine D1-like Receptors and Reward-related Incentive Learning , 1998, Neuroscience & Biobehavioral Reviews.

[110]  B. Balleine,et al.  Consciousness—the interface between affect and cognition , 1998 .

[111]  N. Schmajuk,et al.  Occasion setting: Associative learning and cognition in animals. , 1998 .

[112]  K. Berridge,et al.  What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? , 1998, Brain Research Reviews.

[113]  Matías López,et al.  Sensitivity of instrumental responses to an upshift in water deprivation , 1999 .

[114]  E. Gibson,et al.  Chocolate Craving and Hunger State: Implications for the Acquisition and Expression of Appetite and Food Choice , 1999, Appetite.

[115]  J. Mirenowicz,et al.  Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. , 2000, Behavioral neuroscience.

[116]  K. Berridge,et al.  The hedonic impact and intake of food are increased by midazolam microinjection in the parabrachial nucleus , 2000, Brain Research.

[117]  K. Berridge.  Measuring hedonic impact in animals and infants: microstructure of affective taste reactivity patterns , 2000, Neuroscience & Biobehavioral Reviews.

[118]  B. Balleine,et al.  The Effect of Lesions of the Insular Cortex on Instrumental Conditioning: Evidence for a Role in Incentive Memory , 2000, The Journal of Neuroscience.

[119]  C. Limebeer,et al.  The antiemetic drug ondansetron intereferes with lithium-induced conditioned rejection reactions, but not lithium induced taste avoidance in rats. , 2000 .

[120]  K. Berridge,et al.  Intra-Accumbens Amphetamine Increases the Conditioned Incentive Salience of Sucrose Reward: Enhancement of Reward “Wanting” without Enhanced “Liking” or Response Reinforcement , 2000, The Journal of Neuroscience.

[121]  K. Berridge.  Reward learning: Reinforcement, incentives, and expectations , 2000 .

[122]  Comparison of the instrumental reinforcer devaluation effect in two strains of rats (Wistar and Lister) , 2000, Behavioural Processes.