The Optimization of Kernel CMAC Based on BYY Learning

Cerebellar Model Articulation Controller (CMAC) has attractive properties of fast learning and simple computation. The kernel CMAC, which provides an interpretation for the classic CMAC from the kernel viewpoint, strengthens the modeling capability without increasing its complexity. However, the kernel CMAC suffers from the problem of selecting its hyperparameter. In this paper, the Bayesian Ying-Yang (BYY) learning theory is incorporated into kernel CMAC, referred to as KCMAC-BYY, to optimize the hyperparameter. The BYY learning is motivated from the well-known Chinese Taoism Yin-Yang philosophy, and has been developed in this past decade as a unified statistical framework for parameter learning, regularization, structural scale selection and architecture design. The proposed KCMAC-BYY achieves the systematic tuning of the hyperparameter, further improving the performance in modeling capability and stability. The experimental results show that the proposed KCMAC-BYY outperforms the existing representative techniques in the research literature.

[1]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machines , 2002 .

[2]  Lei Xu,et al.  RBF nets, mixture experts, and Bayesian Ying-Yang learning , 1998, Neurocomputing.

[3]  James S. Albus,et al.  Data Storage in the Cerebellar Model Articulation Controller (CMAC) , 1975 .

[4]  Chih-Ming Chen,et al.  A self-organizing HCMAC neural-network classifier , 2003, IEEE Trans. Neural Networks.

[5]  Chun-Shin Lin,et al.  CMAC with General Basis Functions , 1996, Neural Networks.

[6]  Lei Xu,et al.  BYY harmony learning, structural RPCL, and topological self-organizing on mixture models , 2002, Neural Networks.

[7]  L. Devroye.  A Course in Density Estimation , 1987 .

[8]  Tamás Szabó,et al.  Kernel CMAC with improved capability , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[9]  Daming Shi,et al.  FCMAC-BYY: Fuzzy CMAC Using Bayesian Ying–Yang Learning , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  Yau-Hwang Kuo,et al.  Hardware implementation of CMAC neural network with reduced storage requirement , 1997, IEEE Trans. Neural Networks.

[11]  Daming Shi,et al.  An online Bayesian Ying-Yang learning applied to fuzzy CMAC , 2008, Neurocomputing.

[12]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[13]  Lei Xu,et al.  Advances on BYY harmony learning: information theoretic perspective, generalized projection geometry, and independent factor autodetermination , 2004, IEEE Transactions on Neural Networks.

[14]  D.A. Handelman,et al.  Theory and development of higher-order CMAC neural networks , 1992, IEEE Control Systems.