Binary Classification Using Linear SVM Pyramidal Tree

This paper presents a linear SVM (Support Vector Machine) Pyramidal Tree (SVMPT) for binary classification tasks. SVMPT is a modified version of SVM based Tree Type Neural Networks (SVMTNN), reported earlier in the literature [1]. Both the algorithms use parameter-less SVM proposed by Mangasarian [2] for learning in each node. While SVMTNN insists on 100 percent training accuracy, linear SVMPT uses predetermined threshold value to determine when to stop adding new nodes. Experimental results on standard binary datasets show that the algorithm has good generalization capability, comparable to linear SVMs. We also present experimental results on extensions of linear SVMPT to multi-class datasets and datasets with fuzzy membership for each datapoint.

[1]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[2]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[3]  Suresh Chandra,et al.  Binary classification by SVM based tree type neural networks , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[4]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[5]  Tai-Yue Wang,et al.  Fuzzy support vector machine for multi-class text categorization , 2007, Inf. Process. Manag..

[6]  Shigeo Abe,et al.  Multiclass Support Vector Machines , 2010 .

[7]  Huan Liu,et al.  Neural-network feature selector , 1997, IEEE Trans. Neural Networks.

[8]  Reshma Khemchandani,et al.  Fast and robust learning through fuzzy linear proximal support vector machines , 2004, Neurocomputing.

[9]  Olvi L. Mangasarian,et al.  Support vector machine classification via parameterless robust linear programming , 2005, Optim. Methods Softw..

[10]  Tamás D. Gedeon,et al.  Exploring constructive cascade networks , 1999, IEEE Trans. Neural Networks.

[11]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[12]  Rudy Setiono,et al.  Feedforward Neural Network Construction Using Cross Validation , 2001, Neural Computation.

[13]  O. Mangasarian,et al.  Robust linear programming discrimination of two linearly inseparable sets , 1992 .

[14]  I. Hatono,et al.  Fuzzy decision trees by fuzzy ID3 algorithm and its application to diagnosis systems , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[15]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[16]  L. P. Ricotti,et al.  A pyramidal delayed perceptron , 1990 .

[17]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[18]  Madan Gopal,et al.  SVM-Based Tree-Type Neural Networks as a Critic in Adaptive Critic Designs for Control , 2007, IEEE Transactions on Neural Networks.

[19]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .