A Regularized Inverse QR Decomposition Based Recursive Least Squares Algorithm for the CMAC Neural Network

The Cerebellar Model Articulation Controller (CMAC) neural network is an associative memory that is biologically inspired by the cerebellum, which is found in the brains of animals. The standard CMAC uses the least mean squares algorithm (LMS) to train the weights. Recently, the recursive least squares (RLS) algorithm was proposed as a superior algorithm for training the CMAC online as it can converge in one epoch, and does not require tuning of a learning rate. However, the RLS algorithm was found to be very computationally demanding. In this work, the RLS computation time is reduced by using an inverse QR decomposition based RLS (IQR-RLS) algorithm which is also parallelized for multi-core CPUs. Furthermore, this work shows how the IQR-RLS algorithm may be regularized which greatly improves the generalization capabilities of the CMAC.

[1]  Maria D. Miranda,et al.  Conventional and Inverse QRD-RLS Algorithms , 2009 .

[2]  Chris J. Harris,et al.  The interpolation capabilities of the binary CMAC , 1993, Neural Networks.

[3]  Ming Li,et al.  A Learning Algorithm of CMAC Based on RLS , 2004, Neural Processing Letters.

[4]  Mitsuo Kawato,et al.  Learning control for a closed loop system using feedback-error-learning , 1990, 29th IEEE Conference on Decision and Control.

[5]  T. Kailath,et al.  Numerically stable fast transversal filters for recursive least squares adaptive filtering , 1991, IEEE Trans. Signal Process..

[6]  J. Pallotta,et al.  Two dimensional function learning using CMAC neural network with optimized weight smoothing , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[7]  Russell L. Smith,et al.  Intelligent motion control with an artificial cerebellum , 1998 .

[8]  Zonghai Chen,et al.  Continuous CMAC-QRLS and Its Systolic Array , 2005, Neural Processing Letters.

[9]  S. Thomas Alexander,et al.  A Method for Recursive Least Squares Filtering Based Upon an Inverse QR Decomposition , 1993, IEEE Trans. Signal Process..

[10]  James S. Albus,et al.  New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)1 , 1975 .

[11]  Jose A. Apolinario,et al.  Introduction to Adaptive Filters , 2009 .

[12]  Tamás Szabó,et al.  Kernel CMAC with improved capability , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[13]  Mitsuo Kawato,et al.  Learning Control of a Closed Loop System Using Feedback-Error-Learning , 1991 .