Theory and development of higher-order CMAC neural networks

The cerebellar model articulation controller (CMAC) neural network is capable of learning nonlinear functions extremely quickly due to the local nature of its weight updating. The rectangular shape of CMAC receptive field functions, however, produces discontinuous (staircase) function approximations without inherent analytical derivatives. The ability to learn both functions and function derivatives is important for the development of many online adaptive filter, estimation, and control algorithms. It is shown that use of B-spline receptive field functions in conjunction with more general CMAC weight addressing schemes allows higher-order CMAC neural networks to be developed that can learn both functions and function derivatives. This also allows hierarchical and multilayer CMAC network architectures to be constructed that can be trained using standard error back-propagation learning techniques.<<ETX>>

[1]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[2]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[3]  M. Cox.  Curve Fitting with Piecewise Polynomials , 1971 .

[4]  James S. Albus,et al.  New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)1 , 1975 .

[5]  P. J. Hartley,et al.  Tensor Product Approximations to Data Defined on Rectangular Meshes in N-Space , 1976, Comput. J..

[6]  Larry Carter,et al.  Universal Classes of Hash Functions , 1979, J. Comput. Syst. Sci..

[7]  M. G. Cox,et al.  Practical spline approximation , 1982 .

[8]  J. Friedman,et al.  Multidimensional Additive Spline Approximation , 1983 .

[9]  R.M. Dunn,et al.  Brains, behavior, and robotics , 1983, Proceedings of the IEEE.

[10]  Ernest L. Hall,et al.  Intelligent Robots and Computer Vision VI , 1987 .

[11]  D. Sparks,et al.  Sensory and motor maps in the mammalian superior colliculus , 1987, Trends in Neurosciences.

[12]  W. T. Miller,et al.  Shape Recognition Using A CMAC Based Learning System , 1988, Other Conferences.

[13]  Yann LeCun,et al.  A theoretical framework for back-propagation , 1988 .

[14]  John E. Moody,et al.  Fast Learning in Multi-Resolution Hierarchies , 1988, NIPS.

[15]  J. J. Gelfand,et al.  Development of adaptive B-splines using CMAC neural networks , 1989, International 1989 Joint Conference on Neural Networks.

[16]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[17]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[18]  Stephen H. Lane,et al.  Multi-Layer Perceptrons with B-Spline Receptive Field Functions , 1990, NIPS.

[19]  W. T. Miller,et al.  Rapid learning using CMAC neural networks: real time control of an unstable system , 1990, Proceedings. 5th IEEE International Symposium on Intelligent Control 1990.

[20]  D.A. Handelman,et al.  Integrating neural networks and knowledge-based systems for intelligent robotic control , 1990, IEEE Control Systems Magazine.

[21]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..