2009 Special Issue: Intelligence in the brain: A theory of how it works and how to build it

This paper presents a theory of how general-purpose learning-based intelligence is achieved in the mammal brain, and how we can replicate it. It reviews four generations of ever more powerful general-purpose learning designs in Adaptive, Approximate Dynamic Programming (ADP), which includes reinforcement learning as a special case. It reviews empirical results which fit the theory, and suggests important new directions for research, within the scope of NSF's recent initiative on Cognitive Optimization and Prediction. The appendices suggest possible connections to the realms of human subjective experience, comparative cognitive neuroscience, and new challenges in electric power. The major challenge before us today in mathematical neural networks is to replicate the "mouse level", but the paper does contain a few thoughts about building, understanding and nourishing levels of general intelligence beyond the mouse.

[1]  Fung Yu-lan.  The Spirit of Chinese Philosophy , 1947 .

[2]  E. Feigenbaum,et al.  Computers and Thought , 1963 .

[3]  M. Nicolelis,et al.  Induction of immediate spatiotemporal changes in thalamic networks by peripheral block of ascending cutaneous information , 1993, Nature.

[4]  N. Spruston,et al.  Action potential initiation and backpropagation in neurons of the mammalian CNS , 1997, Trends in Neurosciences.

[5]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[6]  D.B. Fogel,et al.  A self-learning evolutionary chess program , 2004, Proceedings of the IEEE.

[7]  Gursel Serpen,et al.  Theoretical Exploration on Local Stability of Simul taneous Recurrent Neural Network Dynamics for Static Combinatorial Optimization , 2004 .

[8]  Paul J. Werbos,et al.  Supervised Learning: Can it Escape its Local Minimum? , 1994 .

[9]  D. Yankelovich,et al.  Ego and instinct;: The psychoanalytic view of human nature--revised, , 1970 .

[10]  Paul J. Werbos,et al.  Building and Understanding Adaptive Systems: A Statistical/Numerical Approach to Factory Automation and Brain Research , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[11]  Allen M. Waxman,et al.  A neural system for behavioral conditioning of mobile robots , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[12]  K. Pribram,et al.  Freud's Project reassessed , 1976 .

[13]  James S. Albus,et al.  Outline for a theory of intelligence , 1991, IEEE Trans. Syst. Man Cybern..

[14]  T. Bliss,et al.  Transsynaptic expression of a presynaptic glutamate receptor during hippocampal long-term potentiation. , 1993, Science.

[15]  M. Nicolelis,et al.  Sensorimotor encoding by synchronous neural ensemble activity at multiple levels of the somatosensory system. , 1995, Science.

[16]  Jennie Si,et al.  ADP: Goals, Opportunities and Principles , 2004 .

[17]  P.J. Werbos,et al.  Using ADP to Understand and Replicate Brain Intelligence: the Next Level Design , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[18]  Paul J. Werbos,et al.  The Roots of Backpropagation: From Ordered Derivatives to Neural Networks and Political Forecasting , 1994 .

[19]  Karen Horney,et al.  The Neurotic Personality of Our Time , 1937 .

[20]  V. Kůrková,et al.  Dealing with complexity : a neural networks approach , 1998 .

[21]  P. Werbos.  Bell’s Theorem, Many Worlds and Backwards-Time Physics: Not Just a Matter of Interpretation , 2008, 0801.1234.

[22]  F. Kozin,et al.  System Modeling and Optimization , 1982 .

[23]  S. Kaplan.  The Physiology of Thought , 1950 .

[24]  Kevin Warwick,et al.  A Brain-Like Design to Learn Optimal Decision Strategies in Complex Environments , 1998 .

[25]  Michael C. Fu,et al.  Stochastic optimization using model reference adaptive search , 2005, Proceedings of the Winter Simulation Conference, 2005..

[26]  Paul J. Werbos,et al.  Applications of advances in nonlinear sensitivity analysis , 1982 .

[27]  G. C. Quarton,et al.  The Neurosciences;: Second study program , 1970 .

[28]  S. Laberge,et al.  Exploring the World of Lucid Dreaming , 1990 .

[29]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[30]  C. Jung,et al.  The Portable Jung , 1971 .

[31]  Wesley R. Elsberry,et al.  Optimality in Biological and Artificial Networks , 1997 .

[32]  G. Simpson.  The meaning of evolution : a study of the history of life and of its significance for man , 1949 .

[33]  Y. Le Cun,et al.  Comparing different neural network architectures for classifying handwritten digits , 1989, International 1989 Joint Conference on Neural Networks.

[34]  Donald C. Wunsch,et al.  Coordinated machine learning and decision support for situation awareness , 2009, Neural Networks.

[35]  Frank L. Lewis,et al.  A dynamic recurrent neural-network-based adaptive observer for a class of nonlinear systems , 1997, Autom..

[36]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[37]  Jun Zhang,et al.  Adaptive learning via selectionism and Bayesianism, Part II: The sequential case , 2009, Neural Networks.

[38]  Daniel S. Levine,et al.  2009 Special Issue: Brain pathways for cognitive-emotional decision making in the human animal , 2009 .

[39]  David E. Goldberg,et al.  Hierarchical Bayesian Optimization Algorithm , 2006, Scalable Optimization via Probabilistic Modeling.

[40]  Paul J. Werbos,et al.  Putting more brain-like intelligence into the electric power grid: What we need and how to do it , 2009, 2009 International Joint Conference on Neural Networks.

[41]  Jennie Si,et al.  Handbook of Learning and Approximate Dynamic Programming (IEEE Press Series on Computational Intelligence) , 2004 .

[42]  Robert Kozma,et al.  Beyond Feedforward Models Trained by Backpropagation: A Practical Training Tool for a More Efficient Universal Approximator , 2007, IEEE Transactions on Neural Networks.

[43]  Leonid I. Perlovsky,et al.  Language and cognition , 2009, Neural Networks.

[44]  M. Bitterman.  THE EVOLUTION OF INTELLIGENCE. , 1965, Scientific American.

[45]  Warren B. Powell,et al.  Handbook of Learning and Approximate Dynamic Programming , 2006, IEEE Transactions on Automatic Control.

[46]  Stefan Schaal,et al.  Memory-based neural networks for robot learning , 1995, Neurocomputing.

[47]  Robert Kozma,et al.  The KIV model of intentional dynamics and decision making , 2009, Neural Networks.

[48]  K. Siu,et al.  Theoretical Advances in Neural Computation and Learning , 1994, Springer US.

[49]  Enrico Gobbetti,et al.  Encyclopedia of Electrical and Electronics Engineering , 1999 .

[50]  Richard S. Sutton,et al.  Neural networks for control , 1990 .

[51]  J. Mallet,et al.  Characterization of a presynaptic glutamate receptor. , 1993, Science.