Learning Controllers for Complex Behavioral Systems.

Abstract : Biological control systems routinely guide complex dynamical systems through complicated tasks such as running or diving. Conventional control techniques, however, stumble with these problems, which have complex dynamics, many degrees of freedom, and a task which is often only partially specified. To address problems like these, we are using a biologically inspired, hierarchical control structure, in which controllers composed of radial basis function networks learn the controls required at each level of the hierarchy. Through learning and proper encoding of behaviors and controls, some of these difficulties in controlling complex systems can be overcome.

[1]  Geoffrey J. Gordon.  Stable Function Approximation in Dynamic Programming , 1995, ICML.

[2]  Salvatore Monaco,et al.  Digital control through finite feedback discretizability , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[3]  S. Shankar Sastry,et al.  Biological motor control approaches for a planar diver , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[4]  Jessica K. Hodgins,et al.  Animation of Human Diving , 1996, Comput. Graph. Forum.

[5]  H. Harry Asada,et al.  Recursive experimental structure re-design of a robot arm using rapid prototyping , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[6]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[7]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[8]  I. Kolmanovsky,et al.  Controllability of a class of nonlinear systems with drift , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[9]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[10]  G. Gottlieb,et al.  Strategies for the control of voluntary movements with one mechanical degree of freedom , 1989, Behavioral and Brain Sciences.

[11]  Michael I. Jordan,et al.  MASSACHUSETTS INSTITUTE OF TECHNOLOGY ARTIFICIAL INTELLIGENCE LABORATORY and CENTER FOR BIOLOGICAL AND COMPUTATIONAL LEARNING DEPARTMENT OF BRAIN AND COGNITIVE SCIENCES , 1996 .

[12]  R. Brockett.  System Theory on Group Manifolds and Coset Spaces , 1972 .

[13]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[14]  W. J. Beek,et al.  A Dynamical Systems Approach to Skill Acquisition , 1992, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[15]  Andrew A. Goldenberg,et al.  Radial basis function network architecture for nonholonomic motion planning and control of free-flying manipulators , 1996, IEEE Trans. Robotics Autom..

[16]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[17]  Roger W. Brockett,et al.  On the computer control of movement , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[18]  Andrew W. Moore,et al.  Generalization in Reinforcement Learning: Safely Approximating the Value Function , 1994, NIPS.

[19]  R. Schmidt.  A schema theory of discrete motor skill learning. , 1975 .

[20]  Roger W. Brockett.  Analog and Digital Computing , 1992, 25th Anniversary of INRIA.

[21]  Steven M. Finbeiner.  The Neural and Behavioral Organization of Goal-Directed Movements , 1989, The Yale Journal of Biology and Medicine.

[22]  Ilya Kolmanovsky,et al.  Developments in nonholonomic control problems , 1995 .

[23]  John N. Tsitsiklis,et al.  Neuro-dynamic programming: an overview , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[24]  C. Frohlich.  Do springboard divers violate angular momentum conservation , 1979 .

[25]  Charles A. Batterman,et al.  The Techniques Of Springboard Diving , 1977 .

[26]  S. Shankar Sastry,et al.  Path planning for nonholonomic systems with drift , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[27]  John N. Tsitsiklis,et al.  Asynchronous stochastic approximation and Q-learning , 1994, Mach. Learn..

[28]  Jessica K. Hodgins,et al.  Biped Gymnastics , 1988, Int. J. Robotics Res..

[29]  S. Grillner.  Locomotion in vertebrates: central mechanisms and reflex interaction. , 1975, Physiological reviews.