Kernel CMAC with improved capability

Cerebellar model articulation controller (CMAC) neural network is a real alternative to MLP and RBF and has some advantageous features: its training is fast and its architecture is especially suitable for digital hardware implementation. The price of these attractive features is its rather poor capability. CMAC may have significant approximation and generalization errors. The generalization error can be significantly reduced if a regularization term is applied during training, while the approximation capability can be improved if the complexity of the network is increased. The paper shows that using a kernel interpretation the approximation capability of the network can be improved without increasing the complexity. It also shows, that regularization can be applied in the kernel representation too, so a new version of the CMAC is proposed where both approximation and generalization capabilities are improved significantly.

[1]  Carl de Boor,et al.  A Practical Guide to Splines , 1978, Applied Mathematical Sciences.

[2]  B. Southern,et al.  The inverse of a semi-infinite symmetric banded matrix , 1997 .

[3]  John E. Moody,et al.  Fast Learning in Multi-Resolution Hierarchies , 1988, NIPS.

[4]  Gábor Horváth,et al.  CMAC : RECONSIDERING AN OLD NEURAL NETWORK , 2003 .

[5]  Aníbal R. Figueiras-Vidal,et al.  Generalizing CMAC architecture and training , 1998, IEEE Trans. Neural Networks.

[6]  Shih-Lin Hung,et al.  MS_CMAC Neural Network Learning Model in Structural Engineering , 1999 .

[7]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[8]  Shun-Feng Su,et al.  Credit assigned CMAC and its application to online learning robust controllers , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[9]  Shih-Lin Hung,et al.  High-order MS CMAC neural network , 2001, IEEE Trans. Neural Networks.

[10]  G. Horvath,et al.  FPGA implementation of the kernel CMAC , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[11]  James S. Albus,et al.  New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)1 , 1975 .

[12]  G. Horváth,et al.  A generalised LS-SVM , 2003 .

[13]  W. T. Miller,et al.  CMAC: an associative neural network alternative to backpropagation , 1990, Proc. IEEE.

[14]  Luo Zhong,et al.  The unfavourable effects of hash coding on CMAC convergence and compensatory measure , 1997, 1997 IEEE International Conference on Intelligent Processing Systems (Cat. No.97TH8335).

[15]  Chun-Shin Lin,et al.  A low-dimensional-CMAC-based neural network , 1996, 1996 IEEE International Conference on Systems, Man and Cybernetics. Information Intelligence and Systems (Cat. No.96CH35929).

[16]  Chris J. Harris,et al.  The interpolation capabilities of the binary CMAC , 1993, Neural Networks.

[17]  Ming Li,et al.  A Learning Algorithm of CMAC Based on RLS , 2004, Neural Processing Letters.

[18]  Yau-Hwang Kuo,et al.  Hardware implementation of CMAC neural network with reduced storage requirement , 1997, IEEE Trans. Neural Networks.

[19]  J. L. Schiano,et al.  Hash-coding in CMAC neural networks , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[20]  Tamás Szabó,et al.  Improving the generalization capability of the binary CMAC , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[21]  Yau-Hwang Kuo,et al.  Hardware realization of higher-order CMAC model for color calibration , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[22]  Russell L. Smith,et al.  Intelligent motion control with an artificial cerebellum , 1998 .

[23]  Hongjun Lu,et al.  Fuzzy System and CMAC Network with B-Spline Membership/Basis Functions can Approximate A Smooth Function and its Derivatives , 2003, Int. J. Comput. Intell. Appl..

[24]  James S. Albus,et al.  Data Storage in the Cerebellar Model Articulation Controller (CMAC) , 1975 .

[25]  T. Szabó,et al.  Cmac and Its Extensions for Efficient System Modelling , 1999 .

[26]  Chun-Shin Lin,et al.  Neural networks with self-organized basis functions , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[27]  Chih-Ming Chen,et al.  A self-organizing HCMAC neural-network classifier , 2003, IEEE Trans. Neural Networks.

[28]  Chun-Shin Lin,et al.  Learning convergence of CMAC technique , 1997, IEEE Trans. Neural Networks.

[29]  Chun-Shin Lin,et al.  CMAC with General Basis Functions , 1996, Neural Networks.

[30]  Chun-Shin Lin,et al.  A new neural network structure composed of small CMACs , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[31]  C.-S. Lin,et al.  Use of adaptive resolution for better CMAC learning , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[32]  G. Horvath,et al.  A sparse least squares support vector machine classifier , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[33]  J. Pallotta,et al.  Two dimensional function learning using CMAC neural network with optimized weight smoothing , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[34]  P. C. Parks,et al.  Convergence Properties of Associative Memory Storage for Learning Control Systems , 1990 .

[35]  Alexander Gammerman,et al.  Ridge Regression Learning Algorithm in Dual Variables , 1998, ICML.

[36]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[37]  D.A. Handelman,et al.  Theory and development of higher-order CMAC neural networks , 1992, IEEE Control Systems.

[38]  Ching-Tsan Chiang,et al.  Neural networks composed of single-variable CMACs , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[39]  R. Shah,et al.  Least Squares Support Vector Machines , 2022 .

[40]  Feng Qian,et al.  Fuzzy CMAC and its application , 2000, Proceedings of the 3rd World Congress on Intelligent Control and Automation (Cat. No.00EX393).

[41]  David E. Thompson,et al.  Neighborhood sequential and random training techniques for CMAC , 1995, IEEE Trans. Neural Networks.

[42]  Cheng-Jian Lin,et al.  A self-organizing recurrent fuzzy CMAC model for dynamic system identification , 2004, FUZZ-IEEE.

[43]  Gabor Horvzith.  Improving the generalization capability of the binary CMAC , 2000 .

[44]  D. Ellison,et al.  On the Convergence of the Multidimensional Albus Perceptron , 1991, Int. J. Robotics Res..

[45]  Zonghai Chen,et al.  Continuous CMAC-QRLS and Its Systolic Array , 2005, Neural Processing Letters.

[46]  A. J. Simons.  COMMENT: Analytical inversion of a particular type of banded matrix , 1997 .

[47]  Ching-Chang Wong,et al.  A fuzzy CMAC structure and learning method for function approximation , 2001, 10th IEEE International Conference on Fuzzy Systems. (Cat. No.01CH37297).