Localized Support Vector Machines for Classification

Support vector machines (SVMs) have been promising methods in pattern recognition because of their solid mathematical foundation. In this paper, we propose a localized SVM classification scheme (LSVM). In which we first cluster the training data in each category, and then train a set of SVMs based on these dusters. The SVMs trained from the clusters in each category that are nearest to the given input pattern are then selected for the final classification. Our experiments on six UCI datasets show that LSVM outperforms the traditional SVM.

[1]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[2]  O. Mangasarian,et al.  Robust linear programming discrimination of two linearly inseparable sets , 1992 .

[3]  Léon Bottou,et al.  Local Learning Algorithms , 1992, Neural Computation.

[4]  Andrew W. Moore,et al.  X-means: Extending K-means with Efficient Estimation of the Number of Clusters , 2000, ICML.

[5]  Stefan Rüping,et al.  Incremental Learning with Support Vector Machines , 2001, ICDM.

[6]  Daphne Koller,et al.  Support Vector Machine Active Learning with Applications to Text Classification , 2000, J. Mach. Learn. Res..

[7]  Osamu Watanabe,et al.  A Random Sampling Technique for Training Support Vector Machines , 2001, ALT.

[8]  Y.H. Chen,et al.  Cluster-based support vector machines in text-independent speaker identification , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[9]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[10]  Zheng Nanning,et al.  Unsupervised clustering based reduced support vector machines , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[11]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[12]  Antônio de Pádua Braga,et al.  SVM-KM: speeding SVMs learning with a priori cluster selection and k-means , 2000, Proceedings. Vol.1. Sixth Brazilian Symposium on Neural Networks.

[13]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[14]  Inderjit S. Dhillon,et al.  Kernel k-means: spectral clustering and normalized cuts , 2004, KDD.

[15]  Jiawei Han,et al.  Classifying large data sets using SVMs with hierarchical clusters , 2003, KDD '03.

[16]  Deepak K. Agarwal,et al.  Shrinkage estimator generalizations of Proximal Support Vector Machines , 2002, KDD.

[17]  Chih-Jen Lin,et al.  Training v-Support Vector Classifiers: Theory and Algorithms , 2001, Neural Computation.

[18]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[19]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[20]  Pavel Berkhin,et al.  A Survey of Clustering Data Mining Techniques , 2006, Grouping Multidimensional Data.