Tikhonov training of the CMAC neural network

The architecture of the cerebellar model articulation controller (CMAC) presents a rigid compromise between learning and generalization. In the presence of a sparse training dataset, this limitation manifestly causes overfitting, a drawback that is not overcome by current training algorithms. This paper proposes a novel training framework founded on the Tikhonov regularization, which relates to the minimization of the power of the sigma-order derivative. This smoothness criterion yields to an internal cell-interaction mechanism that increases the generalization beyond the degree hardcoded in the CMAC architecture while preserving the potential CMAC learning capabilities. The resulting training mechanism, which proves to be simple and computationally efficient, is deduced from a rigorous theoretical study. The performance of the new training framework is validated against comparative benchmarks from the DELVE environment

[1]  Lutz Prechelt,et al.  A quantitative study of experimental evaluations of neural network learning algorithms: Current research practice , 1996, Neural Networks.

[2]  Tamás Szabó,et al.  Kernel CMAC with improved capability , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[3]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[4]  Luis Weruaga,et al.  Frequency domain formulation of active parametric deformable models , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Chun-Shin Lin,et al.  A sum-of-product neural network (SOPNN) , 2000, Neurocomputing.

[6]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[7]  Christopher M. Bishop,et al.  Current address: Microsoft Research, , 2022 .

[8]  Demetri Terzopoulos,et al.  United Snakes , 1999, Medical Image Anal..

[9]  David E. Rumelhart,et al.  Generalization by Weight-Elimination with Application to Forecasting , 1990, NIPS.

[10]  Lin Chun-Shin,et al.  CMAC with General Basis Functions. , 1996, Neural networks : the official journal of the International Neural Network Society.

[11]  David E. Thompson,et al.  Neighborhood sequential and random training techniques for CMAC , 1995, IEEE Trans. Neural Networks.

[12]  Mohamad T. Musavi,et al.  On the Generalization Ability of Neural Network Classifiers , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  W. T. Miller,et al.  CMAC: an associative neural network alternative to backpropagation , 1990, Proc. IEEE.

[14]  Chris J. Harris,et al.  The interpolation capabilities of the binary CMAC , 1993, Neural Networks.

[15]  Aníbal R. Figueiras-Vidal,et al.  Generalizing CMAC architecture and training , 1998, IEEE Trans. Neural Networks.

[16]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[17]  Yasuhiko Morimoto,et al.  Efficient Construction of Regression Trees with Range and Region Splitting , 1997, Machine Learning.

[18]  Vladimir Cherkassky,et al.  Factors controlling generalization ability of MLP networks , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[19]  James S. Albus,et al.  New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)1 , 1975 .

[20]  Rafael Verdú,et al.  Convergence analysis of active contours in image segmentation , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[21]  Christopher M. Bishop,et al.  Curvature-driven smoothing: a learning algorithm for feedforward networks , 1993, IEEE Trans. Neural Networks.

[22]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[23]  James S. Albus,et al.  I A New Approach to Manipulator Control: The I Cerebellar Model Articulation Controller , 1975 .

[24]  Piero Zamperoni,et al.  Plus ça va, moins ça va , 1996, Pattern Recognit. Lett..

[25]  Chao He,et al.  Learning Convergence of CMAC Algorithm , 2004, Neural Processing Letters.

[26]  Hyongsuk Kim,et al.  Selection of learning parameters for CMAC-based adaptive critic learning , 1995, IEEE Trans. Neural Networks.

[27]  Antonio Artés-Rodríguez,et al.  Fourier analysis of the generalized CMAC neural network , 1998, Neural Networks.

[28]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[29]  Chun-Shin Lin,et al.  A new neural network structure composed of small CMACs , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[30]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[31]  J. Pallotta,et al.  Two dimensional function learning using CMAC neural network with optimized weight smoothing , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).