Kernel machines and additive fuzzy systems: classification and function approximation

This paper investigates the connection between additive fuzzy systems and kernel machines. We prove that, under quite general conditions, these two seemingly quite distinct models are essentially equivalent. As a result, algorithms based upon support vector (SV) learning are proposed to build fuzzy systems for classification and function approximation. The performance of the proposed algorithm is illustrated using extensive experimental results.

[1]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[2]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[3]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[4]  Peter L. Bartlett,et al.  For Valid Generalization the Size of the Weights is More Important than the Size of the Network , 1996, NIPS.

[5]  Bart Kosko,et al.  The shape of fuzzy sets in adaptive function approximation , 2001, IEEE Trans. Fuzzy Syst..

[6]  Olvi L. Mangasarian,et al.  Generalized Support Vector Machines , 1998 .

[7]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[8]  Chuen-Tsai Sun,et al.  Neuro-fuzzy modeling and control , 1995, Proc. IEEE.

[9]  Marc G. Genton,et al.  Classes of Kernels for Machine Learning: A Statistics Perspective , 2002, J. Mach. Learn. Res..

[10]  Hao Ying,et al.  General SISO Takagi-Sugeno fuzzy systems with linear rule consequent are universal approximators , 1998, IEEE Trans. Fuzzy Syst..

[11]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[12]  Linda Kaufman,et al.  Solving the quadratic programming problem arising in support vector classification , 1999 .

[13]  M. Sugeno,et al.  Structure identification of fuzzy model , 1988 .

[14]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[15]  Bart Kosko,et al.  Fuzzy function approximation with ellipsoidal rules , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[16]  John Yen,et al.  Fuzzy Logic - A Modern Perspective , 1999, IEEE Trans. Knowl. Data Eng..

[17]  Riccardo Rovatti,et al.  Fuzzy piecewise multilinear and piecewise linear systems as universal approximators in Sobolev norms , 1998, IEEE Trans. Fuzzy Syst..

[18]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[19]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[20]  Hung-Yuan Chung,et al.  A self-learning fuzzy logic controller using genetic algorithms with reinforcements , 1997, IEEE Trans. Fuzzy Syst..

[22]  Shigeo Abe,et al.  Function approximation based on fuzzy rules extracted from partitioned numerical data , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[23]  D. Dubois,et al.  Operations on fuzzy numbers , 1978 .

[24]  Magne Setnes,et al.  Supervised fuzzy clustering for rule extraction , 1999, FUZZ-IEEE'99. 1999 IEEE International Fuzzy Systems. Conference Proceedings (Cat. No.99CH36315).

[25]  Bernhard Schölkopf,et al.  The connection between regularization operators and support vector kernels , 1998, Neural Networks.

[26]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[27]  Bernhard Schölkopf,et al.  Improving the Accuracy and Speed of Support Vector Machines , 1996, NIPS.

[28]  Robert Babuska,et al.  Rule base reduction: some comments on the use of orthogonal transforms , 2001, IEEE Trans. Syst. Man Cybern. Syst..

[29]  John Yen,et al.  Application of statistical information criteria for optimal fuzzy model construction , 1998, IEEE Trans. Fuzzy Syst..