Theory of functional systems, adaptive critics and neural networks

We propose a general scheme of intelligent adaptive control system based on the Petr K. Anokhin's theory of functional systems. This scheme is aimed at controlling adaptive purposeful behavior of an animat (a simulated animal) that has several natural needs (e.g., energy replenishment, reproduction). The control system consists of a set of hierarchically linked functional systems and enables predictive and goal-directed behavior. Each functional system includes a neural network based adaptive critic design. We also discuss schemes of prognosis, decision making, action selection and learning that occur in the functional systems and in the whole control system of the animat.

[1]  Mikhail S. Burtsev,et al.  Measuring the Dynamics of Artificial Evolution , 2003, ECAL.

[2]  Kevin Warwick,et al.  A Brain-Like Design to Learn Optimal Decision Strategies in Complex Environments , 1998 .

[3]  程东峰,et al.  Wilson病与肝移植 , 2004 .

[4]  Richard S. Sutton,et al.  Reinforcement learning architectures for animats , 1991 .

[5]  Paul J. Werbos,et al.  Approximate dynamic programming for real-time control and neural modeling , 1992 .

[6]  Bernard Widrow,et al.  Punish/Reward: Learning with a Critic in Adaptive Threshold Systems , 1973, IEEE Trans. Syst. Man Cybern..

[7]  Aude Billard,et al.  From Animals to Animats , 2004 .

[8]  Stanley J. Rosenschein,et al.  From Animals to Animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior , 1996 .

[9]  Jean-Arcady Meyer,et al.  Learning reactive and planning rules in a motivationally autonomous animat , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[10]  W. T. Powers.  Behavior, the control of perception , 1973 .

[11]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[12]  G. Edelman.  Neural Darwinism: The Theory Of Neuronal Group Selection , 1989 .

[13]  P. K. Anokhin,et al.  After Pavlov. (Book Reviews: Biology and Neurophysiology of the Conditioned Reflex and Its Role in Adaptive Behavior) , 2013 .

[14]  W. Gantt.  Biology and Neurophysiology of the Conditioned Reflex and Its Role in Adaptive Behavior , 1976 .

[15]  Stewart W. Wilson.  The animat path to AI , 1991 .

[16]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.