Kernel Ho-Kashyap classifier with generalization control

This paper introduces a new classifier design method based on a kernel extension of the classical Ho-Kashyap procedure. The proposed method uses an approximation of the absolute error rather than the squared error to design a classifier, which leads to robustness against outliers and a better approximation of the misclassification error. Additionally, easy control of the generalization ability is obtained using the structural risk minimization induction principle from statistical learning theory. Finally, examples are given to demonstrate the validity of the introduced method.

[1]  L. Mirsky,et al.  The Theory of Matrices , 1961, The Mathematical Gazette.

[2]  R. L. Kashyap,et al.  An Algorithm for Linear Inequalities and its Applications , 1965, IEEE Trans. Electron. Comput..

[3]  Y. Ho,et al.  A Class of Iterative Procedures for Linear Inequalities , 1966 .

[4]  Peter Lancaster,et al.  The theory of matrices , 1969 .

[5]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[6]  Julius T. Tou,et al.  Pattern Recognition Principles , 1974 .

[7]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[8]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[9]  Kenneth Rose,et al.  A global optimization technique for statistical classifier design , 1996, IEEE Trans. Signal Process..

[10]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[11]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[12]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[13]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[14]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[15]  Gunnar Rätsch,et al.  Input space versus feature space in kernel-based methods , 1999, IEEE Trans. Neural Networks.

[16]  Jacek M. Leski,et al.  Fuzzy and Neuro-Fuzzy Intelligent Systems , 2000, Studies in Fuzziness and Soft Computing.

[17]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[18]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[19]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[20]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[21]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[22]  Jacek M. Leski,et al.  Ho-Kashyap classifier with generalization control , 2003, Pattern Recognit. Lett..

[23]  J. Wade Davis,et al.  Statistical Pattern Recognition , 2003, Technometrics.

[24]  Jacek M. Łȩski,et al.  Ho--Kashyap classifier with generalization control , 2003 .

[25]  Jacek Łęski,et al.  A fuzzy if-then rule-based nonlinear classifier , 2003 .

[26]  Gunnar Rätsch,et al.  Soft Margins for AdaBoost , 2001, Machine Learning.

[27]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.