On Learning in a Time-Varying Environment by Using a Probabilistic Neural Network and the Recursive Least Squares Method

This paper presents the recursive least squares method, combined with the general regression neural networks, applied to solve the problem of learning in time-varying environment. The general regression neural network is based on the orthogonal-type kernel functions. The appropriate algorithm is presented in a recursive form. Sufficient simulations confirm empirically the convergence of the algorithm.

[1]  Janusz T. Starczewski,et al.  Connectionist Structures of Type 2 Fuzzy Inference Systems , 2001, PPAM.

[2]  L. Rutkowski,et al.  A neuro-fuzzy controller with a compromise fuzzy reasoning , 2002 .

[3]  L. Rutkowski.  On-line identification of time-varying systems by nonparametric techniques , 1982 .

[4]  Leszek Rutkowski,et al.  Neural Networks and Soft Computing , 2003 .

[5]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[6]  L. Rutkowski.  Sequential Estimates of a Regression Function by Orthogonal Series with Applications in Discrimination , 1981 .

[7]  L. Rutkowski.  Nonparametric identification of quasi-stationary systems , 1985 .

[8]  Ewaryst Rafaj⌈owicz.  Nonparametric orthogonal series estimators of regression: A class attaining the optimal convergence rate in L2☆ , 1987 .

[9]  Ryszard Tadeusiewicz,et al.  Artificial Intelligence and Soft Computing - ICAISC 2006, 8th International Conference, Zakopane, Poland, June 25-29, 2006, Proceedings , 2006, International Conference on Artificial Intelligence and Soft Computing.

[10]  Leszek Rutkowski.  Nonparametric Procedures for Identification and Control of Linear Dynamic Systems , 1988 .

[11]  L. Rutkowski.  Non-parametric learning algorithms in time-varying environments☆ , 1989 .

[12]  L. Rutkowski.  Application of multiple Fourier series to identification of multivariable non-stationary systems , 1989 .

[13]  L. Györfi,et al.  A Distribution-Free Theory of Nonparametric Regression (Springer Series in Statistics) , 2002 .

[14]  Leszek Rutkowski,et al.  A general approach to neuro-fuzzy systems , 2001, 10th IEEE International Conference on Fuzzy Systems. (Cat. No.01CH37297).

[15]  P. J. Green,et al.  Probability and Statistical Inference , 1978 .

[16]  Leszek Rutkowski,et al.  A new method for system modelling and pattern classification , 2004 .

[17]  Leszek Rutkowski.  Multiple Fourier series procedures for extraction of nonlinear regressions from noisy data , 1993, IEEE Trans. Signal Process..

[18]  Janusz T. Starczewski,et al.  Interval Type 2 Neuro-Fuzzy Systems Based on Interval Consequents , 2003 .

[19]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[20]  Leszek Rutkowski,et al.  A fast training algorithm for neural networks , 1998 .

[21]  Leszek Rutkowski.  On Bayes Risk Consistent Pattern Recognition Procedures in a Quasi-Stationary Environment , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Krzysztof Patan,et al.  Optimal training strategies for locally recurrent neural networks , 2011 .

[23]  E. Rafajłowicz,et al.  On optimal global rate of convergence of some nonparametric identification procedures , 1989 .

[24]  R. Nowicki.  Nonlinear modelling and classification based on the MICOG defuzzification , 2009 .

[25]  Leszek Rutlowski.  Sequential pattern recognition procedures derived from multiple Fourier series , 1988 .

[26]  L. Rutkowski,et al.  Nonparametric recovery of multivariate functions with applications to system identification , 1985, Proceedings of the IEEE.

[27]  W. Greblicki,et al.  An orthogonal series estimate of time-varying regression , 1983 .

[28]  Rene F. Swarttouw,et al.  Orthogonal polynomials , 2020, NIST Handbook of Mathematical Functions.

[29]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[30]  Leszek Rutkowski,et al.  On system identification by nonparametric function fitting , 1982 .

[31]  W. T. Federer,et al.  Stochastic Approximation and NonLinear Regression , 2003 .

[32]  Marcin Korytkowski,et al.  Modular Type-2 Neuro-fuzzy Systems , 2007, PPAM.

[33]  W. Greblicki,et al.  Density-free Bayes risk consistency of nonparametric pattern recognition procedures , 1981, Proceedings of the IEEE.

[34]  Erwin Kreyszig,et al.  Introductory Mathematical Statistics. , 1970 .

[35]  Leszek Rutkowski,et al.  A general approach for nonparametric fitting of functions and their derivatives with applications to linear circuits identification , 1986 .

[36]  L. Rutkowski,et al.  Nonparametric fitting of multivariate functions , 1986 .

[37]  Leszek Rutkowski,et al.  Identification of MISO nonlinear regressions in the presence of a wide class of disturbances , 1991, IEEE Trans. Inf. Theory.

[38]  L. Rutkowski.  Real-time identification of time-varying systems by non-parametric algorithms based on Parzen kernels , 1985 .

[39]  Robert Cierniak,et al.  On image compression by competitive neural networks and optimal linear predictors , 2000, Signal Process. Image Commun..

[40]  L. Rutkowski.  On nonparametric identification with prediction of time-varying systems , 1984 .

[41]  Rafal Scherer.  Boosting Ensemble of Relational Neuro-fuzzy Systems , 2006, ICAISC.

[42]  Leszek Rutkowski,et al.  ORTHOGONAL SERIES ESTIMATES OF A REGRESSION FUNCTION WITH APPLICATIONS IN SYSTEM IDENTIFICATION , 1982 .