A Theory of Conditioning: Inductive Learning Within

We present a theory of classical conditioning based on a parallel, rule-based performance system integrated with mechanisms for inductive learning. Inferential heuristics are used to add new rules to the system in response to the relation between the system's predictions and environmental input. A major heuristic is based on "unusuamess": Novel cues are favored as candidates to predict events that are important or unexpected. Rules have strength values that are revised on the basis of feedback. The performance system allows rules to operate in parallel, competing to control behavior and obtain reward for successful prediction of important events. Sets of rules can form default hierarchies: Exception rules censor useful but imperfect default rules, protecting them from loss of strength. The theory is implemented as a computer simulation, which is used to model a broad range of conditioning phenomena, including blocking and overshadowing, the impact of statistical predictability on conditioning, and conditioned inhibition. The theory accounts for a variety of phenomena that previous theories have not dealt with successfully.

[1]  P. Holland.  Second-order conditioning with and without unconditioned stimulus presentation. , 1980, Journal of experimental psychology. Animal behavior processes.

[2]  Richard Hirsh,et al.  The hippocampus, conditional operations, and cognition , 1980 .

[3]  J. Konorski.  Conditioned reflexes and neuron organization. , 1948 .

[4]  S. Revusky.  Chapter 4 – THE ROLE OF INTERFERENCE IN ASSOCIATION OVER A DELAY1 , 1971 .

[5]  R. Rescorla,et al.  Extinction of Pavlovian conditioned inhibition. , 1974, Journal of comparative and physiological psychology.

[6]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[7]  R. Hogarth,et al.  Judging probable cause. , 1986 .

[8]  R. Rescorla.  Probability of shock in the presence and absence of CS in fear conditioning. , 1968, Journal of comparative and physiological psychology.

[9]  Ralph R. Miller,et al.  Recovery of an overshadowed association achieved by extinction of the overshadowing stimulus. , 1985 .

[10]  L. Squire.  The neuropsychology of human memory. , 1982, Annual review of neuroscience.

[11]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[12]  N. Mackintosh,et al.  One-Trial Overshadowing , 1979 .

[13]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[14]  N. Mackintosh,et al.  Surprise and blocking: Effects of the number of compound trials , 1980 .

[15]  M. Seligman.  On the generality of the laws of learning , 1970 .

[16]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[17]  P. Holland,et al.  Extinction of Inhibition after Serial and Simultaneous Feature Negative Discrimination Training , 1986, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[18]  A. G. Baker,et al.  Excitatory and inhibitory conditioning following uncorrelated presentations of CS and UCS , 1977 .

[19]  R. Rescorla.  Summation and retardation tests of latent inhibition. , 1971, Journal of comparative and physiological psychology.

[20]  H. Kelley.  The processes of causal attribution. , 1973 .

[21]  A. G. Baker.  Learned Irrelevance and Learned Helplessness: Rats Learn that Stimuli, Reinforcers, and Responses are Uncorrelated. , 1976 .

[22]  H. Fowler,et al.  Effects of contingency violations on the extinction of a conditioned fear inhibitor and a conditioned fear excitor. , 1986, Journal of experimental psychology. Animal behavior processes.

[23]  L. Kamin.  Attention-like processes in classical conditioning , 1967 .

[24]  N. Mackintosh,et al.  Conditioning And Associative Learning , 1983 .

[25]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[26]  J. Konorski.  Integrative activity of the brain : an interdisciplinary approach , 1967 .

[27]  L. Kamin,et al.  Compound conditioned emotional response conditioning with differentially salient elements in rats. , 1974, Journal of comparative and physiological psychology.

[28]  J. Pearce,et al.  A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980 .

[29]  E. Kremer.  The Rescorla-Wagner model: losses in associative strength in compound conditioned stimuli. , 1978, Journal of experimental psychology. Animal behavior processes.

[30]  E. Kremer.  Truly random and traditional control procedures in CER conditioning in the rat. , 1971, Journal of comparative and physiological psychology.

[31]  W. B. Orr,et al.  Hippocampectomy selectively disrupts discrimination reversal conditioning of the rabbit nictitating membrane response , 1983, Behavioural Brain Research.

[32]  A. Dickinson.  Appetitive-aversion interactions: Facilitation of aversive conditioning by prior appetitive training in the rat , 1976 .

[33]  R. Rescorla.  Pavlovian conditioned inhibition , 1969 .

[34]  K. Haberlandt,et al.  Stimulus selection in animal discrimination learning. , 1968, Journal of experimental psychology.

[35]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[36]  R. Hirsh.  The hippocampus and contextual retrieval of information from memory: a theory. , 1974, Behavioral biology.

[37]  A G Barto,et al.  Toward a modern theory of adaptive networks: expectation and prediction. , 1981, Psychological review.

[38]  R. Rescorla.  Evidence for "unique stimulus" account of configural conditioning. , 1973 .

[39]  R. Rescorla.  Variation in the effectiveness of reinforcement and nonreinforcement following prior inhibitory conditioning , 1971 .

[40]  R. Rescorla.  "Configural" conditioning in discrete-trial bar pressing. , 1972, Journal of comparative and physiological psychology.

[41]  P. Solomon,et al.  Latent inhibition and stimulus generalization of the classically conditioned nictitating membrane response in rabbits (Oryctolagus cuniculus) following dorsal hippocampal ablation. , 1975, Journal of comparative and physiological psychology.

[42]  P. W. Frey,et al.  Model of conditioning incorporating the Rescorla-Wagner associative axiom, a dynamic attention process, and a catastrophe rule. , 1978 .

[43]  N. Mackintosh.  Blocking of conditioned suppression: role of the first compound trial. , 1975, Journal of experimental psychology. Animal behavior processes.

[44]  N. Mackintosh.  The psychology of animal learning , 1974 .

[45]  J. Pearce.  A model for stimulus generalization in Pavlovian conditioning. , 1987, Psychological review.

[46]  J. Pearce,et al.  Latent inhibition of a CS during CS-US pairings. , 1979, Journal of experimental psychology. Animal behavior processes.

[47]  W. B. Orr,et al.  Hippocampectomy disrupts acquisition and retention of learned conditional responding. , 1984, Behavioral neuroscience.

[48]  N. J. Mackintosh,et al.  An Analysis of Overshadowing and Blocking , 1971 .

[49]  P. Winston.  Learning by Augmenting Rules and Accumulating Censors. , 1982 .

[50]  R. A. Koelling,et al.  Cues: Their Relative Effectiveness as a Function of the Reinforcer , 1968, Science.

[51]  J. Pearce,et al.  Loss of associability by a compound stimulus comprising excitatory and inhibitory elements. , 1979, Journal of experimental psychology. Animal behavior processes.

[52]  Allen Newell,et al.  Production Systems: Models of Control Structures , 1973 .

[53]  D. M. Green,et al.  Detection and recognition. , 1978 .

[54]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[55]  Anthony Dickinson,et al.  Loss of Associability by a Conditioned Inhibitor* , 1982, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[56]  R. Rescorla.  Informational Variables in Pavlovian Conditioning , 1972 .

[57]  P. Solomon.  Role of the hippocampus in blocking and conditioned inhibition of the rabbit's nictitating membrane response. , 1977, Journal of comparative and physiological psychology.

[58]  Eugene Galanter,et al.  Handbook of mathematical psychology: I. , 1963 .

[59]  B. Skinner,et al.  Principles of Behavior , 1944 .

[60]  A. G. Baker,et al.  Preexposure to the CS alone, US alone, or CS and US uncorrelated: Latent inhibition, blocking by context or learned irrelevance? , 1979 .

[61]  Frederick Mosteller,et al.  Stochastic Models for Learning , 1956 .

[62]  R. Rescorla,et al.  Stimulus similarity as a determinant of Pavlovian conditioning. , 1977, Journal of experimental psychology. Animal behavior processes.

[63]  N. J. Mackintosh,et al.  Stimulus selection: Learning to ignore stimuli that predict no change in reinforcement. , 1973 .

[64]  N. Mackintosh,et al.  Reinforcer specificity in the enhancement of conditioning by posttrial surprise. , 1979 .

[65]  W. K. Honig,et al.  Cognitive Processes in Animal Behavior , 1979 .

[66]  N. Mackintosh.  A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement , 1975 .

[67]  C. B. Woodbury.  The learning of stimulus patterns by dogs. , 1943 .

[68]  M. Domjan,et al.  Backward conditoning as an inhibitory procedure , 1971 .

[69]  R. Bolles,et al.  A nonassociative aspect of overshadowing , 1981 .

[70]  A. R. Wagner,et al.  One-trial overshadowing: evidence of distributive processing. , 1980, Journal of experimental psychology. Animal behavior processes.

[71]  D. R. Williams,et al.  Associative factors underlying the pigeon's key pecking in auto-shaping procedures. , 1973, Journal of the experimental analysis of behavior.

[72]  P. Holland,et al.  Behavioral Studies of Associative Learning in Animals , 1982 .

[73]  L. Squire.  Memory and Brain , 1987 .

[74]  B. Schwartz.  Psychology of Learning and Behavior , 1978 .

[75]  R. E. Lubow,et al.  The Context Effect: The Relationship Between Stimulus Preexposure and Environmental Preexposure Determines Subsequent Learning. , 1976 .

[76]  A. G. Baker.  Conditioned inhibition is not the symmetrical opposite of conditioned excitation: A test of the Rescorla-Wagner model , 1974 .