Toward a modern theory of adaptive networks: expectation and prediction.

Many adaptive neural network theories are based on neuronlike adaptive elements that can behave as single unit analogs of associative conditioning. In this article we develop a similar adaptive element, but one which is more closely in accord with the facts of animal learning theory than elements commonly studied in adaptive network research. We suggest that an essential feature of classical conditioning that has been largely overlooked by adaptive network theorists is its predictive nature. The adaptive element we present learns to increase its response rate in anticipation of increased stimulation, producing a conditioned response before the occurrence of the unconditioned stimulus. The element also is in strong agreement with the behavioral data regarding the effects of stimulus context, since it is a temporally refined extension of the Rescorla-Wagner model. We show by computer simulation that the element becomes sensitive to the most reliable, nonredundant, and earliest predictors of reinforcement . We also point out that the model solves many of the stability and saturation problems encountered in network simulations. Finally, we discuss our model in light of recent advances in the physiology and biochemistry of synaptic mechanisms.

[1]  E. Hilgard The nature of the conditioned response: I. The case for and against stimulus-substitution. , 1936 .

[2]  W. N. Kellogg Evidence for both stimulus-substitution and original anticipatory responses in the conditioning of dogs. , 1938 .

[3]  B. Skinner,et al.  Principles of Behavior , 1944 .

[4]  E. Hilgard,et al.  Theories of Learning , 1981 .

[5]  K. Lashley The problem of serial order in behavior , 1951 .

[6]  John H. Holland,et al.  Tests on a cell assembly theory of the action of the brain, using a large digital computer , 1956, IRE Trans. Inf. Theory.

[7]  P M MILNER,et al.  The cell assembly: Mark II. , 1957, Psychological review.

[8]  J. K. Hawkins Self-Organizing Systems-A Review and Commentary , 1961, Proceedings of the IRE.

[9]  Marvin Minsky,et al.  Steps toward Artificial Intelligence , 1995, Proceedings of the IRE.

[10]  M. D. Egger,et al.  Secondary reinforcement in rats as a function of information value and reliability of the stimulus. , 1962, Journal of experimental psychology.

[11]  E. Feigenbaum,et al.  Computers and Thought , 1963 .

[12]  I. Gormezano,et al.  CONDITIONING OF THE NICTITATING MEMBRANE OF THE RABBIT AS A FUNCTION OF CS-US INTERVAL. , 1964, Journal of comparative and physiological psychology.

[13]  N. Schneiderman Interstimulus interval function of the nictitating membrane response of the rabbit under delay versus trace conditioning. , 1966 .

[14]  R. Rescorla,et al.  Two-process learning theory: Relationships between Pavlovian conditioning and instrumental learning. , 1967, Psychological review.

[15]  L. Kamin Predictability, surprise, attention, and conditioning , 1967 .

[16]  D. Marr A theory of cerebellar cortex , 1969, The Journal of physiology.

[17]  B. Campbell,et al.  Punishment and aversive behavior , 1969 .

[18]  S. Grossberg Some Networks That Can Learn, Remember, and Reproduce any Number of Complicated Space-Time Patterns, I , 1969 .

[19]  G S Brindley,et al.  Nerve net models of plausible size that perform many simple learning tasks , 1969, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[20]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[21]  S. R. Coleman,et al.  Classical conditioning of the rabbit's nictitating membrane response at backward, simultaneous, and forward CS-US intervals. , 1969, Journal of comparative and physiological psychology.

[22]  A. M. Uttley The informon: a network for adaptive pattern recognition. , 1970, Journal of theoretical biology.

[23]  D. N. Spinelli OCCAM: A COMPUTER MODEL FOR A CONTENT ADDRESSABLE MEMORY IN THE CENTRAL NERVOUS SYSTEM , 1970 .

[24]  A. H. Klopf,et al.  Brain Function and Adaptive Systems: A Heterostatic Theory , 1972 .

[25]  J Nagumo,et al.  [A model of associative memory]. , 1972, Iyo denshi to seitai kogaku. Japanese journal of medical electronics and biological engineering.

[26]  Marvin Minsky,et al.  Artificial Intelligence Progress Report , 1972 .

[27]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[28]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[29]  Kaoru Nakano,et al.  Associatron-A Model of Associative Memory , 1972, IEEE Trans. Syst. Man Cybern..

[30]  W. F. Prokasy,et al.  Classical conditioning II: Current research and theory. , 1972 .

[31]  G. Stent A physiological mechanism for Hebb's postulate of learning. , 1973, Proceedings of the National Academy of Sciences of the United States of America.

[32]  P. Greengard,et al.  Advances in Cyclic Nucleotide Research , 1973 .

[33]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[34]  N. Mackintosh The psychology of animal learning , 1974 .

[35]  R. Rescorla,et al.  Extinction of Pavlovian conditioned inhibition. , 1974, Journal of comparative and physiological psychology.

[36]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[37]  A. M. Uttley The informon in classical conditioning. , 1975, Journal of theoretical biology.

[38]  B. Libet,et al.  Synaptic coupling into the production and storage of a neuronal memory trace , 1975, Nature.

[39]  Donald O. Walter,et al.  Mass action in the nervous system , 1975 .

[40]  George E. P. Box,et al.  Time Series Analysis: Forecasting and Control , 1977 .

[41]  Raymond J. Bandlow Theories of Learning, 4th Edition. By Ernest R. Hilgard and Gordon H. Bower. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1975 , 1976 .

[42]  F. F. Weight,et al.  Modulation of synaptic transmitter release by repetitive postsynaptic action potentials. , 1976, Science.

[43]  P. Greengard Possible role for cyclic nucleotides and phosphorylated membrane proteins in postsynaptic actions of neurotransmitters† , 1976, Nature.

[44]  A. M. Uttley Neurophysiological predictions of a two-pathway informon theory of neural conditioning , 1976, Brain Research.

[45]  A. M. Uttley A two-pathway informon theory of conditioning and adaptive pattern recognition , 1976, Brain Research.

[46]  A. M. Uttley Simulation studies of learning in an informon network , 1976, Brain Research.

[47]  Stephen A. Ritz,et al.  Distinctive features, categorical perception, and probability learning: some applications of a neural model , 1977 .

[48]  Shun-ichi Amari,et al.  A Mathematical Approach to Neural Systems , 1977 .

[49]  B. Cohen Cellular Basis of Behavior , 1977, Neurology.

[50]  Teuvo Kohonen,et al.  Associative memory. A system-theoretical approach , 1977 .

[51]  J. Nathanson Cyclic nucleotides and nervous system function. , 1977, Physiological reviews.

[52]  P. W. Frey,et al.  Model of conditioning incorporating the Rescorla-Wagner associative axiom, a dynamic attention process, and a catastrophe rule. , 1978 .

[53]  C. Woody If Cyclic GMP is a Neuronal Second Messenger What is the Message , 1978 .

[54]  E. John,et al.  The neurophysiology of information processing and cognition. , 1978, Annual review of psychology.

[55]  A. Dickinson,et al.  Classical conditioning in animals. , 1978, Annual review of psychology.

[56]  Eric R. Kandel,et al.  A cell-biological approach to learning , 1978 .

[57]  D. Jenden Cholinergic Mechanisms and Psychopharmacology , 1978, Advances in Behavioral Biology.

[58]  P. A. Smith,et al.  Synaptic potentials in sympathetic ganglia: are they mediated by cyclic nucleotides? , 1978, Science.

[59]  C. C. Wood Variations on a theme by Lashley: lesion experiments on the neural model of Anderson, Silverstein, Ritz, and Jones. , 1978, Psychological review.

[60]  D. Alkon Voltage-dependent calcium and potassium ion conductances: a contingency mechanism for an associative learning model. , 1979, Science.

[61]  J. Albus Mechanisms of planning and problem solving in the brain , 1979 .

[62]  W. F. Prokasy,et al.  The effect of US omission in classical aversive and appetitive conditioning of rabbits , 1979 .

[63]  F. F. Weight,et al.  Long-lasting synaptic potentials and the modulation of synaptic transmission. , 1979, Federation proceedings.

[64]  A. M. Uttley,et al.  Information transmission in the nervous system , 1979 .

[65]  W. F. Hill,et al.  Mechanisms of Learning and Motivation: A Memorial Volume to Jerzy Konorski , 1980 .

[66]  A. Markoe Advances in Cyclic Nucleotide Research , 1982 .

[67]  Stephen Grossberg,et al.  Classical and Instrumental Learning by Neural Networks , 1982 .

[68]  John S. Edwards,et al.  The Hedonistic Neuron: A Theory of Memory, Learning and Intelligence , 1983 .