Wavelets in identification wavelets, splines, neurons, fuzzies : how good for identification

This is a tutorial about nonparametric nonlinear system identification. Advantages and limitations of this approach are discussed from the engineer's point of view. Classical as well as modern techniques are discussed, this includes kernel and projection estimates, neural networks and hinging hyperplanes, and mainly wavelet estimators. Both practical and mathematical issues are investigated. Advantages and limitations of wavelet based techniques are emphazised. Finally we show how fuzzy models may play a role in this game, as a framework for expressing prior knowledge on the system. The whole material is illustrated on some application examples.

[1]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[2]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[3]  J. Morgan,et al.  Problems in the Analysis of Survey Data, and a Proposal , 1963 .

[4]  E. Nadaraya.  On Estimating Regression , 1964 .

[5]  G. S. Watson,et al.  Smooth regression analysis , 1964 .

[6]  N. Draper,et al.  Applied Regression Analysis. , 1967 .

[7]  T. Wagner,et al.  Asymptotically Optimal Discriminant for Pattern Classification Functions , 1969 .

[8]  H. Akaike.  Statistical predictor identification , 1970 .

[9]  Louis A. Hageman,et al.  Iterative Solution of Large Linear Systems. , 1971 .

[10]  Sándor Csibi,et al.  Stochastic processes with learning properties , 1975 .

[11]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[12]  L. Devroye,et al.  Distribution-Free Consistency Results in Nonparametric Discrimination and Regression Function Estimation , 1980 .

[13]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[14]  Eduardo Sontag.  Nonlinear regulation: The piecewise linear approach , 1981 .

[15]  R. Z. Khasʹminskiĭ,et al.  Statistical estimation : asymptotic theory , 1981 .

[16]  C. J. Stone,et al.  Optimal Global Rates of Convergence for Nonparametric Regression , 1982 .

[17]  Luc Devroye,et al.  Any Discrimination Rule Can Have an Arbitrarily Bad Probability of Error for Finite Sample Size , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  H. Triebel.  Theory Of Function Spaces , 1983 .

[19]  J. Rice.  Bandwidth Choice for Nonparametric Regression , 1984 .

[20]  Luc Devroye,et al.  Nonparametric Density Estimation , 1985 .

[21]  W. Härdle,et al.  Optimal Bandwidth Selection in Nonparametric Regression Function Estimation , 1985 .

[22]  Ker-Chau Li,et al.  Asymptotic optimality of CL and generalized cross-validation in ridge regression with application to spline smoothing , 1986 .

[23]  Michèle Basseville,et al.  The asymptotic local approach to change detection and model validation , 1987 .

[24]  H. Müller,et al.  Variable Bandwidth Kernel Estimators of Regression Curves , 1987 .

[25]  P. Petrushev,et al.  Rational Approximation of Real Functions , 1988 .

[26]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[27]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[28]  K S Narendra,et al.  IDENTIFICATION AND CONTROL OF DYNAMIC SYSTEMS USING NEURAL NETWORKS , 1990 .

[29]  W. Sickel.  Spline Representations of Functions in Besov-Triebel-Lizorkin Spaces on ℝ n , 1990 .

[30]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[31]  Madan M. Gupta,et al.  Conditional Logic in Expert Systems , 1991 .

[32]  David L. Donoho,et al.  Interpolating Wavelet Transforms , 1992 .

[33]  Peter J. Gawthrop,et al.  Neural networks for control systems - A survey , 1992, Autom..

[34]  B. Portier.  Estimation non parametrique et commande adaptative de processus markoviens non lineaires , 1992 .

[35]  G. Kerkyacharian,et al.  Density estimation in Besov spaces , 1992 .

[36]  Qinghua Zhang,et al.  Wavelet networks , 1992, IEEE Trans. Neural Networks.

[37]  R. DeVore,et al.  Compression of wavelet decompositions , 1992 .

[38]  Qinghua Zhang,et al.  Early Warning of Slight Changes in Systems and Plants , 1993 .

[39]  L. Wang,et al.  Fuzzy systems are universal approximators , 1992, [1992 Proceedings] IEEE International Conference on Fuzzy Systems.

[40]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[41]  I. Daubechies.  Ten Lectures on Wavelets , 1992 .

[42]  Leo Breiman,et al.  Hinging hyperplanes for regression, classification, and function approximation , 1993, IEEE Trans. Inf. Theory.

[43]  A. Tsybakov,et al.  Minimax theory of image reconstruction , 1993 .

[44]  Qinghua Zhang,et al.  Monitoring the Combustion Set of a Gas Turbine , 1994 .

[45]  Shie Qian,et al.  Signal representation using adaptive normalized Gaussian functions , 1994, Signal Process..

[46]  I. Johnstone,et al.  Density estimation by wavelet thresholding , 1996 .

[47]  Fuzzy Logic in Control Systems : Fuzzy Logic , 2022 .