Assessment of the Rescorla-Wagner model.

The Rescorla-Wagner model has been the most influential theory of associative learning to emerge from the study of animal behavior over the last 25 years. Recently, equivalence to this model has become a benchmark in assessing connectionist models, with such equivalence often achieved by incorporating the Widrow-Hoff delta rule. This article presents the Rescorla-Wagner model's basic assumptions, reviews some of the model's predictive successes and failures, relates the failures to the model's assumptions, and discusses the model's heuristic value. It is concluded that the model has had a positive influence on the study of simple associative learning by stimulating research and contributing to new model development. However, this benefit should neither lead to the model being regarded as inherently "correct" nor imply that its predictions can be profitably used to assess other models.

[1]  C. H. Honzik,et al.  Degrees of hunger, reward and non-reward, and maze learning in rats, and Introduction and removal of reward, and maze performance in rats , 1930 .

[2]  D. Whitteridge Lectures on Conditioned Reflexes , 1942, Nature.

[3]  C. B. Woodbury The learning of stimulus patterns by dogs. , 1943 .

[4]  C. L. Hull Principles of Behavior , 1945 .

[5]  J. Konorski Conditioned reflexes and neuron organization. , 1948 .

[6]  R. R. Bush,et al.  A Mathematical Model for Simple Learning , 1951 .

[7]  W. Kessen Response strength and conditioned stimulus intensity. , 1953, Journal of Experimental Psychology.

[8]  D. A. Grant,et al.  Eyelid conditioning as a function of unconditioned stimulus intensity and intertrial interval. , 1958, Journal of experimental psychology.

[9]  R. Lubow,et al.  Latent inhibition: the effect of nonreinforced pre-exposure to the conditional stimulus. , 1959, Journal of comparative and physiological psychology.

[10]  D. R. Thomas,et al.  The effects of delayed testing on generalization slope. , 1962, Journal of comparative and physiological psychology.

[11]  I. Gormezano,et al.  Effects of alternating classical conditioning and extinction sessions on the conditioned nictitating membrane response of the rabbit , 1965 .

[12]  Peter Secretan Learning , 1965, Mental Health.

[13]  John Garcia,et al.  Relation of cue to consequence in avoidance learning , 1966 .

[14]  J. Konorski Integrative activity of the brain , 1967 .

[15]  R. Rescorla Probability of shock in the presence and absence of CS in fear conditioning. , 1968, Journal of comparative and physiological psychology.

[16]  K. Haberlandt,et al.  Stimulus selection in animal discrimination learning. , 1968, Journal of Experimental Psychology.

[17]  P. W. Frey,et al.  Classical conditioning of the rabbit eyelid response as a function of interstimulus interval. , 1968, Journal of comparative and physiological psychology.

[18]  R. Rescorla Pavlovian conditioned inhibition , 1969 .

[19]  R. Rescorla Conditioned inhibition of fear resulting from negative CS-US contingencies. , 1969, Journal of comparative and physiological psychology.

[20]  R. Rescorla Reduction in the effectiveness of reinforcement after prior excitatory conditioning , 1970 .

[21]  R. Rescorla Variation in the effectiveness of reinforcement and nonreinforcement following prior inhibitory conditioning , 1971 .

[22]  P Dunham,et al.  Learning and performance. , 1971, Research quarterly.

[23]  N. Mackintosh,et al.  Blocking as a Function of Novelty of CS and Predictability of UCS , 1971, The Quarterly journal of experimental psychology.

[24]  E Gamzu,et al.  Classical Conditioning of a Complex Skeletal Response , 1971, Science.

[25]  E. Kremer Truly random and traditional control procedures in CER conditioning in the rat. , 1971, Journal of comparative and physiological psychology.

[26]  R. Rescorla A theory of pavlovian conditioning: The effectiveness of reinforcement and non-reinforcement , 1972 .

[27]  R. Rescorla,et al.  Associations in second-order conditioning and sensory preconditioning. , 1972, Journal of comparative and physiological psychology.

[28]  M. Domjan,et al.  Specificity of cue to consequence in aversion learning in the rat , 1972 .

[29]  R. Rescorla Evidence for "unique stimulus" account of configural conditioning. , 1973 .

[30]  F. Hoehler,et al.  The effects of overtraining and successive extinctions upon nictitating membrane conditioning in the rabbit , 1973 .

[31]  A. F. Carr Latent inhibition and overshadowing in conditioned emotional response conditioning with rats. , 1974, Journal of comparative and physiological psychology.

[32]  N. Mackintosh The psychology of animal learning , 1974 .

[33]  I. Gormezano,et al.  CS Intensity effects on rabbit nictitating membrane conditioning, extinction and generalization , 1974, The Pavlovian journal of biological science.

[34]  E. Wasserman STIMULUS-REINFORCER PREDICTIVENESS AND SELECTIVE DISCRIMINATION LEARNING IN PIGEONS ' , 1974 .

[35]  A. G. Baker Conditioned inhibition is not the symmetrical opposite of conditioned excitation: A test of the Rescorla-Wagner model , 1974 .

[36]  L. Levitan Tests of the Rescorla-Wagner model of Pavlovian conditioning , 1975 .

[37]  D. Blough Steady state data and a quantitative model of operant generalization and discrimination. , 1975 .

[38]  M. A. Saavedra Pavlovian compound conditioning in the rabbit , 1975 .

[39]  R. Rescorla,et al.  Reinstatement of fear to an extinguished conditioned stimulus. , 1975, Journal of experimental psychology. Animal behavior processes.

[40]  N. Mackintosh A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement , 1975 .

[41]  R. Rescorla,et al.  Second-order conditioning with food unconditioned stimulus. , 1975, Journal of comparative and physiological psychology.

[42]  R. Rescorla Second-order conditioning of Pavlovian conditioned inhibition , 1976 .

[43]  C. F. Cegavske,et al.  Tone-induced changes in excitability of abducens motoneurons and of the reflex path of nictitating membrane response in rabbit (Oryctolagus cuniculus). , 1976, Journal of comparative and physiological psychology.

[44]  R. Rescorla Stimulus generalization: some predictions from a model of Pavlovian conditioning. , 1976, Journal of experimental psychology. Animal behavior processes.

[45]  N. Mackintosh Overshadowing and stimulus intensity , 1976, Animal learning & behavior.

[46]  A. Tomie Interference with autoshaping by prior context conditioning. , 1976 .

[47]  A. G. Baker Learned Irrelevance and Learned Helplessness: Rats Learn that Stimuli, Reinforcers, and Responses are Uncorrelated. , 1976 .

[48]  N. Mackintosh,et al.  Surprise and the attenuation of blocking. , 1976 .

[49]  N. Mackintosh,et al.  Loss of control by a less valid or by a less salient stimulus compounded with a better predictor of reinforcement , 1977 .

[50]  A. G. Baker,et al.  Excitatory and inhibitory conditioning following uncorrelated presentations of CS and UCS , 1977 .

[51]  D. Gillan,et al.  Taste-aversion conditioning with expected versus unexpected drug treatment. , 1977, Journal of experimental psychology. Animal behavior processes.

[52]  P. W. Frey,et al.  Model of conditioning incorporating the Rescorla-Wagner associative axiom, a dynamic attention process, and a catastrophe rule. , 1978 .

[53]  A. Dickinson,et al.  Classical conditioning in animals. , 1978, Annual review of psychology.

[54]  E. Kremer The Rescorla-Wagner model: losses in associative strength in compound conditioned stimuli. , 1978, Journal of experimental psychology. Animal behavior processes.

[55]  N. Mackintosh,et al.  CLASSICAL CONDITIONING IN ANIMALSl , 1978 .

[56]  N. Mackintosh,et al.  One-Trial Overshadowing , 1979 .

[57]  A. G. Baker,et al.  Preexposure to the CS alone, US alone, or CS and US uncorrelated: Latent inhibition, blocking by context or learned irrelevance? , 1979 .

[58]  V. Lolordo,et al.  Preconditioning exposure to the unconditined stimulus affects the acquisition of a conditioned emotional response , 1979 .

[59]  N. Mackintosh,et al.  Reinforcer specificity in the enhancement of conditioning by posttrial surprise. , 1979 .

[60]  E. Kremer Effect of posttrial episodes on conditioning in compound conditioned stimuli. , 1979, Journal of experimental psychology. Animal behavior processes.

[61]  R. F. Westbrook,et al.  Potentiation instead of overshadowing in the pigeon. , 1979, Behavioral and neural biology.

[62]  R. Bolles,et al.  Role of conditioned contextual stimuli in reinstatement of extinguished fear. , 1979, Journal of experimental psychology. Animal behavior processes.

[63]  R. Rescorla,et al.  Potentiation rather than overshadowing in flavor-aversion learning: an analysis in terms of within-compound associations. , 1980, Journal of experimental psychology. Animal behavior processes.

[64]  R. Rescorla Simultaneous and successive associations in sensory preconditioning. , 1980, Journal of experimental psychology. Animal behavior processes.

[65]  J. Walkenbach,et al.  The Rescorla-Wagner Theory of Conditioning: A Review of the Literature , 1980 .

[66]  A. R. Wagner,et al.  One-trial overshadowing: evidence of distributive processing. , 1980, Journal of experimental psychology. Animal behavior processes.

[67]  J. E. Mazur,et al.  Evaluation of Blocking and Conditioned Inhibition to a CS Signaling a Decrease in US Intensity , 1980 .

[68]  J. Pearce,et al.  A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980 .

[69]  K. Rusiniak,et al.  Flavor-illness aversions: the peculiar roles of odor and taste in memory for poison. , 1980, Science.

[70]  W. F. Hill,et al.  Mechanisms of Learning and Motivation: A Memorial Volume to Jerzy Konorski , 1980 .

[71]  R. Lubow,et al.  Conditioned Attention Theory1 , 1981 .

[72]  H. M. Jenkins,et al.  Contingency in fear conditioning: A reexamination , 1981 .

[73]  H. M. Jenkins,et al.  Responses eliminated by noncontingent or negatively contingent reinforcement recover in extinction. , 1981, Journal of experimental psychology. Animal behavior processes.

[74]  A G Barto,et al.  Toward a modern theory of adaptive networks: expectation and prediction. , 1981, Psychological review.

[75]  W. Bellingham,et al.  Attenuation of overshadowing as a function of nondifferential compound conditioning trials , 1981 .

[76]  M. Rashotte,et al.  Signaling functions of the second-order CS: Partial reinforcement during second-order conditioning of the pigeon’s keypeck , 1981 .

[77]  J. Pearce,et al.  The Effects of a Signal for free or for Earned Reward: Implications for the Role of Response-Reinforcer Associations in Instrumental Performance , 1981 .

[78]  A. Randich The US preexposure phenomenon in the conditioned suppression paradigm: A role for conditioned situational stimuli☆ , 1981 .

[79]  N. Mackintosh,et al.  Overshadowing of a Stimulus–Reinforcer Association by an Instrumental Response , 1981 .

[80]  R. Bolles,et al.  A nonassociative aspect of overshadowing , 1981 .

[81]  Ralph R. Miller,et al.  Blocking as a Retrieval Failure: Reactivation of Associations to a Blocked Stimulus , 1982, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[82]  S. Bottjer Conditioned approach and withdrawal behavior in pigeons: Effects of a novel extraneous stimulus during acquisition and extinction , 1982 .

[83]  N. Mackintosh,et al.  Inhibitory Conditioning Resulting from a Reduction in the Magnitude of Reinforcement , 1982, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[84]  Ralph R. Miller,et al.  Blocking with a single compound trial , 1982 .

[85]  Ralph R. Miller,et al.  Reminder-induced recovery of associations to an overshadowed stimulus , 1982 .

[86]  M. Domjan The principles of learning and behavior , 1982 .

[87]  R. Hinson Effects of UCS preexposure on excitatory and inhibitory rabbit eyelid conditioning: an associative effect of conditioned contextual stimuli. , 1982, Journal of experimental psychology. Animal behavior processes.

[88]  P. Durlach Effect of signaling intertrial unconditioned stimuli in autoshaping. , 1983, Journal of experimental psychology. Animal behavior processes.

[89]  Ralph R. Miller,et al.  Reminder-induced recovery from blocking as a function of the number of compound trials , 1983 .

[90]  J. Overmier,et al.  On unpredictability as a causal factor in “learned helplessness” , 1983 .

[91]  Ralph R. Miller,et al.  Information processing in animals : memory mechanisms , 1983 .

[92]  B. T. Lett Extinction of taste aversion does not eliminate taste potentiation of odor aversion in rats or color aversion in pigeons , 1984 .

[93]  M. Bouton Differential control by context in the inflation and reinstatement paradigms. , 1984 .

[94]  Extinction of taste aversion does not eliminate taste potentiation of odor aversion in pigeons , 1984 .

[95]  P. DeVito Conditioned Inhibition and Extinction , 1984 .

[96]  A. Randich,et al.  Mechanisms of blocking by contextual stimuli , 1984 .

[97]  R. R. Miller,et al.  Blocking but not conditioned inhibition results when an added stimulus is reinforced in compound with multiple pretrained stimuli. , 1985, The American journal of psychology.

[98]  Ralph R. Miller,et al.  Recovery of an overshadowed association achieved by extinction of the overshadowing stimulus. , 1985 .

[99]  Ralph R. Miller,et al.  Information processing in animals : conditioned inhibition , 1985 .

[100]  A. Dickinson,et al.  Contingency Effects with Maintained Instrumental Reinforcement , 1985 .

[101]  Ralph R. Miller,et al.  Reinstatement-induced recovery of a taste-LiCl association following extinction , 1985 .

[102]  N. Spear,et al.  Potentiation and overshadowing in preweanling and adult rats. , 1985, Journal of experimental psychology. Animal behavior processes.

[103]  M. Best,et al.  Characteristics of taste-mediated environmental potentiation in rats , 1985 .

[104]  R. R. Miller,et al.  Retrieval variability: sources and consequences. , 1986, The American journal of psychology.

[105]  M. Bouton Slow reacquisition following the extinction of conditioned suppression , 1986 .

[106]  G. Cicala,et al.  Blocking of conditioned suppression with 1 or 10 compound trials , 1986 .

[107]  J. Overmier,et al.  Within-compound associations modulate the relative effectiveness of differential and Pavlovian conditioned inhibition procedures. , 1986 .

[108]  H. Fowler,et al.  Effects of contingency violations on the extinction of a conditioned fear inhibitor and a conditioned fear excitor. , 1986, Journal of experimental psychology. Animal behavior processes.

[109]  P. Durlach Explicitly unpaired procedure as a response elimination technique in autoshaping. , 1986 .

[110]  R. Tait,et al.  Concurrent development of excitatory and inhibitory associations during backward conditioning , 1986 .

[111]  R. R. Miller,et al.  Associative effects of US preexposure: modulation of conditioned responding by an excitatory training context. , 1987, Journal of experimental psychology. Animal behavior processes.

[112]  J. Pearce A model for stimulus generalization in Pavlovian conditioning. , 1987, Psychological review.

[113]  Kathleen J. Nelson Conditioned Inhibition from Incomplete Reductions in the Probability of Reinforcement , 1987 .

[114]  Ralph R. Miller,et al.  Covariation in conditioned response strength between stimuli trained in compound , 1987 .

[115]  Ralph R. Miller,et al.  Memory for associative history of a conditioned stimulus , 1987 .

[116]  M. Bitterman,et al.  Discrimination of color-odor compounds by honeybees: Tests of a continuity model , 1987 .

[117]  H. M. Jenkins,et al.  Effect of signaling extra unconditioned stimuli on autoshaping , 1987 .

[118]  J. Ayres,et al.  Extending conditioned stimuli before versus after unconditioned stimuli: Implications for real-time models of conditioning. , 1987 .

[119]  R. Rescorla Facilitation and inhibition. , 1987 .

[120]  E. Kehoe,et al.  Temporal primacy overrides prior training in serial compound conditioning of the rabbit’s nictitating membrane response , 1987 .

[121]  R. R. Miller,et al.  The comparator hypothesis of conditioned response generation: manifest conditioned excitation and inhibition as a function of relative excitatory strengths of CS and conditioning context at the time of testing. , 1987, Journal of experimental psychology. Animal behavior processes.

[122]  H. Fowler,et al.  Enhancement of conditioned inhibition via an extinction treatment , 1987 .

[123]  R. R. Miller,et al.  Mechanisms underlying retarded emergence of conditioned responding following inhibitory training: evidence for the comparator hypothesis. , 1987, Journal of experimental psychology. Animal behavior processes.

[124]  T. Devietti,et al.  Latent inhibition: A trace conditioning phenomenon? , 1987 .

[125]  Ralph R. Miller,et al.  The comparator hypothesis: A response rule for the expression of associations. , 1988 .

[126]  G. Cicala,et al.  Increased conditioning in rats to a blocked CS after the first compound trial , 1988 .

[127]  J. Overmier,et al.  Some types of conditioned inhibitors carry collateral excitatory associations , 1988 .

[128]  Ralph R. Miller,et al.  Learned irrelevance exceeds the sum of CS-preexposure and US-preexposure deficits. , 1988 .

[129]  E. Kehoe A layered network model of associative learning: learning to learn and configuration. , 1988, Psychological review.

[130]  Ralph R. Miller,et al.  Conditioned excitation and conditioned inhibition are not mutually exclusive , 1988 .

[131]  N. Spear,et al.  Expression of a taste aversion conditioned with an odor-taste compound: Overshadowing is relatively weak in weanlings and decreases over a retention interval in adults , 1988 .

[132]  A. Markman LMS rules and the inverse base-rate effect: Comment on Gluck and Bower (1988). , 1989 .

[133]  Ralph R. Miller,et al.  Superconditioning and Overshadowing , 1989 .

[134]  J. Overmier,et al.  General learned irrelevance: Proactive effects on Pavlovian conditioning in dogs , 1989 .

[135]  A. R. Wagner,et al.  Evolution of a structured connectionist model of Pavlovian conditioning (AESOP). , 1989 .

[136]  E. Wasserman Attribution of Causality to Common and Distinctive Elements of Compound Stimuli , 1990 .

[137]  Ralph R. Miller,et al.  Excitation and inhibition as a function of posttraining extinction of the excitatory cue used in pavlovian inhibition training , 1990 .

[138]  Richard S. Sutton,et al.  Time-Derivative Models of Pavlovian Reinforcement , 1990 .

[139]  S. Robbins Mechanisms underlying spontaneous recovery in autoshaping. , 1990 .

[140]  P. Balsam,et al.  The effects of varying the interreinforcement interval on appetitive contextual conditioning , 1991 .

[141]  V. Lolordo,et al.  Taste-mediated potentiation of odor aversion induced by lithium choloride: Effects of preconditioning exposure to the conditioned stimulus and postoconditioning extinction of the taste aversion , 1991 .

[142]  Development of intersensory function: age-related differences in stimulus selection of multimodal compounds in rats as revealed by Pavlovian conditioning. , 1991, Journal of experimental psychology. Animal behavior processes.

[143]  J. Pearce,et al.  Effects of extinction with a compound conditioned stimulus , 1991 .

[144]  J. Pearce,et al.  Failure of excitatory conditioning to extinguish the influence of a conditioned inhibitor. , 1991 .

[145]  Effects of extinction with a compound conditioned stimulus , 1991 .

[146]  E. Kehoe,et al.  In the Blink of an Eye: Real-Time Stimulus Factors in Delay and Trace Conditioning of the Rabbit's Nictitating Membrane Response , 1991, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[147]  Ralph R. Miller,et al.  Associative structures underlying enhanced negative summation following operational extinction of a Pavlovian inhibitor , 1992 .

[148]  E. Kehoe,et al.  Rapid reacquisition in conditioning of the rabbit's nictitating membrane response. , 1992 .

[149]  Ralph R. Miller,et al.  Overshadowing-like effects between potential comparator stimuli: Covariation in comparator roles of context and punctate excitor used in inhibitory training as a function of excitor salience☆ , 1992 .

[150]  M. Papini,et al.  Effects of the Number of Trials per Session on Autoshaping in Rats , 1993 .

[151]  M. Bouton Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. , 1993, Psychological bulletin.

[152]  Ralph R. Miller,et al.  Local context and the comparator hypothesis , 1993 .

[153]  V. Lolordo,et al.  Evidence for Simultaneous Excitatory and Inhibitory Associations in the Explicitly Unpaired Procedure , 1994 .

[154]  E. Wasserman,et al.  Cue Competition in Causality Judgments: The Role of Nonpresentation of Compound Stimulus Elements , 1994 .

[155]  N. Mackintosh,et al.  Learned Irrelevance is not the Sum of Exposure to CS and US , 1995, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.