Using ADP to Understand and Replicate Brain Intelligence: the Next Level Design

Since the 1960's the author proposed that we could understand and replicate the highest level of intelligence seen in the brain, by building ever more capable and general systems for adaptive dynamic programming (ADP) - like "reinforcement learning" but based on approximating the Bellman equation and allowing the controller to know its utility function. Growing empirical evidence on the brain supports this approach. Adaptive critic systems now meet tough engineering challenges and provide a kind of first-generation model of the brain. Lewis, Prokhorov and myself have early second-generation work. Mammal brains possess three core capabilities - creativity/imagination and ways to manage spatial and temporal complexity - even beyond the second generation. This paper reviews previous progress, and describes new tools and approaches to overcome the spatial complexity gap.

[1]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[2]  S. Kaplan.  The Physiology of Thought , 1950 .

[3]  E. Feigenbaum,et al.  Computers and Thought , 1963 .

[4]  M. Bitterman.  THE EVOLUTION OF INTELLIGENCE. , 1965, Scientific American.

[5]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[6]  Paul J. Werbos,et al.  Applications of advances in nonlinear sensitivity analysis , 1982 .

[7]  Paul J. Werbos,et al.  Building and Understanding Adaptive Systems: A Statistical/Numerical Approach to Factory Automation and Brain Research , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[8]  Richard S. Sutton,et al.  Neural networks for control , 1990 .

[9]  James S. Albus,et al.  Outline for a theory of intelligence , 1991, IEEE Trans. Syst. Man Cybern..

[10]  Paul J. Werbos,et al.  The Roots of Backpropagation: From Ordered Derivatives to Neural Networks and Political Forecasting , 1994 .

[11]  Paul J. Werbos,et al.  The roots of backpropagation , 1994 .

[12]  Kevin Warwick,et al.  A Brain-Like Design to Learn Optimal Decision Strategies in Complex Environments , 1998 .

[13]  Paul J. Werbos,et al.  Stable adaptive control using new critic designs , 1998, Other Conferences.

[14]  Paul J. Werbos,et al.  What do neural nets and quantum theory tell us about mind and reality , 2002, q-bio/0311006.

[15]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[16]  Robert Kozma,et al.  Cellular SRN Trained by Extended Kalman Filter Shows Promise for ADP , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[17]  Warren B. Powell,et al.  Handbook of Learning and Approximate Dynamic Programming , 2006, IEEE Transactions on Automatic Control.

[18]  P. Werbos.  Backwards Differentiation in AD and Neural Nets: Past Links and New Opportunities , 2006 .