Efficient computations for large least square support vector machine classifiers

We observed that the linear system in the training of the least square support vector machine (LSSVM) proposed by Suykens and Vandewalle (Neural process. Lett. 9 (1999a) 293-300; IEEE Trans. Neural Networks 10(4) (1999b) 907-912) can be placed in a more symmetric form so that for a data set with N data points and m features, the linear system can be solved by inverting an m × m instead of an N × N matrix and storing and working with matrices of size at most m × N. This allows us to apply LSSVM to very large data set with small number of features. Our computations show that a data set with a million data points and 10 features can be trained in only 45 s. We also compared the effectiveness and efficiency of our method to standard LSSVM and standard SVM. An example using a quadratic kernel is also given.

[1]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[2]  D Haussler,et al.  Knowledge-based analysis of microarray gene expression data by using support vector machines. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[3]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[4]  J. Platt.  Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[5]  Johan A. K. Suykens,et al.  Training multilayer perceptron classifiers based on a modified support vector method , 1999, IEEE Trans. Neural Networks.

[6]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[7]  Johan A. K. Suykens,et al.  Nonlinear modeling : advanced black-box techniques , 1998 .

[8]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[9]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[10]  V. Vapnik.  The Support Vector Method of Function Estimation , 1998 .

[11]  David R. Musicant,et al.  Lagrangian Support Vector Machines , 2001, J. Mach. Learn. Res..

[12]  Vladimir Vapnik,et al.  The Support Vector Method , 1997, ICANN.

[13]  Jianchang Mao,et al.  Scaling-up support vector machines using boosting algorithm , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[14]  Padhraic Smyth,et al.  Towards scalable support vector machines using squashing , 2000, KDD '00.

[15]  David W. Aha,et al.  Generalizing from Case studies: A Case Study , 1992, ML.