Adaptive Fuzzy Filtering in a Deterministic Setting

Many real-world applications involve the filtering and estimation of process variables. This study considers the use of interpretable Sugeno-type fuzzy models for adaptive filtering. Our aim in this study is to provide different adaptive fuzzy filtering algorithms in a deterministic setting. The algorithms are derived and studied in a unified way without making any assumptions on the nature of signals (i.e., process variables). The study extends, in a common framework, the adaptive filtering algorithms (usually studied in signal processing literature) and p -norm algorithms (usually studied in machine learning literature) to semilinear fuzzy models. A mathematical framework is provided that allows the development and an analysis of the adaptive fuzzy filtering algorithms. We study a class of nonlinear LMS-like algorithms for the online estimation of fuzzy model parameters. A generalization of the algorithms to the p-norm is provided using Bregman divergences (a standard tool for online machine learning algorithms).

[1]  L. Bregman The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[2]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[3]  Simon Haykin,et al.  Adaptive filter theory (2nd ed.) , 1991 .

[4]  Thomas Kailath,et al.  Hoo Optimality Criteria for LMS and Backpropagation , 1993, NIPS 1993.

[5]  Peter Auer,et al.  Exponentially many local minima for single neurons , 1995, NIPS.

[6]  Manfred K. Warmuth,et al.  Additive versus exponentiated gradient updates for linear prediction , 1995, STOC '95.

[7]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[8]  Ali H. Sayed,et al.  H∞ optimality of the LMS algorithm , 1996, IEEE Trans. Signal Process..

[9]  Philip M. Long,et al.  Worst-case quadratic loss bounds for prediction using linear functions and gradient descent , 1996, IEEE Trans. Neural Networks.

[10]  P. Lindskog Fuzzy identification from a grey box modeling point of view , 1997 .

[11]  E. Mizutani,et al.  Neuro-Fuzzy and Soft Computing-A Computational Approach to Learning and Machine Intelligence [Book Review] , 1997, IEEE Transactions on Automatic Control.

[12]  Manfred K. Warmuth,et al.  Exponentiated Gradient Versus Gradient Descent for Linear Predictors , 1997, Inf. Comput..

[13]  Ferenc Szeifert,et al.  Adaptive fuzzy inference system and its application in modelling and model based control , 1999 .

[14]  Claudio Gentile,et al.  The Robustness of the p-Norm Algorithms , 1999, COLT '99.

[15]  Manfred K. Warmuth,et al.  Relative loss bounds for single neurons , 1999, IEEE Trans. Neural Networks.

[16]  Ali H. Sayed,et al.  A unified approach to the steady-state and tracking analyses of adaptive filters , 2001, IEEE Trans. Signal Process..

[17]  Tareq Y. Al-Naffouri,et al.  Adaptive Filters with Error Nonlinearities: Mean-Square Analysis and Optimum Design , 2001, EURASIP J. Adv. Signal Process..

[18]  Chin-Teng Lin,et al.  Noisy speech processing by recurrently adaptive fuzzy filters , 2001, IEEE Trans. Fuzzy Syst..

[19]  Claudio Gentile,et al.  Adaptive and Self-Confident On-Line Learning Algorithms , 2000, J. Comput. Syst. Sci..

[20]  Paulo Sergio Ramirez,et al.  Fundamentals of Adaptive Filtering , 2002 .

[21]  H. W. ENGLz Regularized Data-Driven Construction of Fuzzy Controllers , 2002 .

[22]  Tareq Y. Al-Naffouri,et al.  Transient analysis of adaptive filters with error nonlinearities , 2003, IEEE Trans. Signal Process..

[23]  Tareq Y. Al-Naffouri,et al.  Transient analysis of data-normalized adaptive filters , 2003, IEEE Trans. Signal Process..

[24]  Norbert Stoll,et al.  Robust Adaptive Fuzzy Identification of Time-Varying Processes with Uncertain Data. Handling Uncertainties in the Physical Fitness Fuzzy Approximation with Real World Medical Data: An Application , 2003, Fuzzy Optim. Decis. Mak..

[25]  Norbert Stoll,et al.  Regularized Adaptation of Fuzzy Inference Systems. Modelling the Opinion of a Medical Expert about Physical Fitness: An Application , 2003, Fuzzy Optim. Decis. Mak..

[26]  Norbert Stoll,et al.  Robust Adaptive Identification of Fuzzy Systems with Uncertain Data , 2004, Fuzzy Optim. Decis. Mak..

[27]  Manfred K. Warmuth,et al.  Relative Loss Bounds for Multidimensional Regression Problems , 1997, Machine Learning.

[28]  D.P. Filev,et al.  An approach to online identification of Takagi-Sugeno fuzzy models , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[29]  Norbert Stoll,et al.  Robust Solution to Fuzzy Identification Problem with Uncertain Data by Regularization , 2004, Fuzzy Optim. Decis. Mak..

[30]  Manfred K. Warmuth,et al.  Relative Loss Bounds for On-Line Density Estimation with the Exponential Family of Distributions , 1999, Machine Learning.

[31]  Takafumi Kanamori,et al.  Information Geometry of U-Boost and Bregman Divergence , 2004, Neural Computation.

[32]  Yoram Singer,et al.  Logistic Regression, AdaBoost and Bregman Distances , 2000, Machine Learning.

[33]  Xiaoou Li,et al.  Fuzzy identification using fuzzy neural networks with stable learning algorithms , 2004, IEEE Transactions on Fuzzy Systems.

[34]  Xin Guo,et al.  On the optimality of conditional expectation as a Bregman predictor , 2005, IEEE Trans. Inf. Theory.

[35]  Meng Joo Er,et al.  Adaptive noise cancellation using enhanced dynamic fuzzy neural networks , 2005, IEEE Transactions on Fuzzy Systems.

[36]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[37]  Norbert Stoll,et al.  An energy-gain bounding approach to robust fuzzy identification , 2006, Autom..

[38]  Norbert Stoll,et al.  A robust design criterion for interpretable fuzzy models with uncertain data , 2006, IEEE Transactions on Fuzzy Systems.

[39]  Frank Nielsen,et al.  On weighting clustering , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Ben Taskar,et al.  Structured Prediction, Dual Extragradient and Bregman Projections , 2006, J. Mach. Learn. Res..

[41]  Babak Hassibi,et al.  The p-norm generalization of the LMS algorithm for adaptive filtering , 2003, IEEE Transactions on Signal Processing.

[42]  N. Stoll,et al.  Deterministic approach to robust adaptive learning of fuzzy models , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[43]  S Kumar,et al.  Handling uncertainties in toxicity modelling using a fuzzy filter , 2007, SAR and QSAR in environmental research.

[44]  K. Thurow,et al.  Robust fuzzy mappings for QSAR studies. , 2007, European journal of medicinal chemistry.

[45]  Matthias Weippert,et al.  Fuzzy Evaluation of Heart Rate Signals for Mental Stress Assessment , 2007, IEEE Transactions on Fuzzy Systems.