Classification through incremental max–min separability

Piecewise linear functions can be used to approximate non-linear decision boundaries between pattern classes. Piecewise linear boundaries are known to provide efficient real-time classifiers. However, they require a long training time. Finding piecewise linear boundaries between sets is a difficult optimization problem. Most approaches use heuristics to avoid solving this problem, which may lead to suboptimal piecewise linear boundaries. In this paper, we propose an algorithm for globally training hyperplanes using an incremental approach. Such an approach allows one to find a near global minimizer of the classification error function and to compute as few hyperplanes as needed for separating sets. We apply this algorithm for solving supervised data classification problems and report the results of numerical experiments on real-world data sets. These results demonstrate that the new algorithm requires a reasonable training time and its test set accuracy is consistently good on most data sets compared with mainstream classifiers.

[1]  Leon Bobrowski,et al.  Design of piecewise linear classifiers from formal neurons by a basis exchange technique , 1991, Pattern Recognit..

[2]  Jack Sklansky,et al.  Pattern Classifiers and Trainable Machines , 1981 .

[3]  Jacek M. Łȩski,et al.  Neuro-fuzzy system with learning tolerant to imprecision , 2003 .

[4]  Adil M. Bagirov,et al.  Max–min separability , 2005, Optim. Methods Softw..

[5]  Alexander E. Kostin,et al.  A simple and fast multi-class piecewise linear pattern classifier , 2006, Pattern Recognit..

[6]  Michal Wozniak,et al.  Algorithm of designing compound recognition system on the basis of combining classifiers with simultaneous splitting feature space into competence areas , 2009, Pattern Analysis and Applications.

[7]  Ludmila I. Kuncheva,et al.  Clustering-and-selection model for classifier combination , 2000, KES'2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516).

[8]  Fritz Wysotzki,et al.  The Piecewise Linear Classifier DIPOL92 , 1994, ECML.

[9]  Xinhua Zhuang,et al.  Piecewise linear classifiers using binary tree structure and genetic algorithm , 1996, Pattern Recognit..

[10]  H. C. Palm.  A new piecewise linear classifier , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[11]  Mineichi Kudo,et al.  Piecewise linear classifiers with an appropriate number of hyperplanes , 1998, Pattern Recognit..

[12]  Gabor T. Herman,et al.  On Piecewise-Linear Classification , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Jack Sklansky,et al.  Locally Trained Piecewise Linear Classifiers , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Zhen-Ping Lo,et al.  Comparison of a neural network and a piecewise linear classifier , 1991, Pattern Recognit. Lett..

[15]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[16]  J. Sklansky,et al.  Automated design of multiple-class piecewise linear classifiers , 1989 .

[17]  Bogdan Raducanu,et al.  Online nonparametric discriminant analysis for incremental subspace learning and recognition , 2008, Pattern Analysis and Applications.

[18]  Adil M. Bagirov,et al.  A Method for Minimization of Quasidifferentiable Functions , 2002, Optim. Methods Softw..

[19]  Jacek M. Leski.  Neuro-fuzzy system with learning tolerant to imprecision , 2003, Fuzzy Sets Syst..

[20]  Huan Liu,et al.  Book review: Machine Learning, Neural and Statistical Classification Edited by D. Michie, D.J. Spiegelhalter and C.C. Taylor (Ellis Horwood Limited, 1994) , 1996, SGAR.

[21]  Jianhua Chen,et al.  An incremental learning algorithm for constructing Boolean functions from positive and negative examples , 2002, Comput. Oper. Res..

[22]  Annabella Astorino,et al.  Polyhedral Separability Through Successive LP , 2002 .

[23]  Adil M. Bagirov.  Minimization Methods for One Class of Nonsmooth Functions and Calculation of Semi-Equilibrium Prices , 1999 .

[24]  Jacek M Leski.  Epsilon-insensitive fuzzy c-regression models: introduction to epsilon-insensitive fuzzy modeling. , 2004, IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society.

[25]  A. Bagirov,et al.  Supervised Data Classification via Max-min Separability , 2005 .

[26]  Jacek M. Leski.  Epsiv-insensitive Fuzzy C-regression Models: Introduction to Epsiv-insensitive Fuzzy Modeling , 2004, IEEE Trans. Syst. Man Cybern. Part B.