Parallel Semiparametric Support Vector Machines

In recent years the number of cores in computers has increased considerably, opening new lines of research to adapt classical techniques of machine learning to a parallel scenario. In this paper, we have developed and implemented with the multi-platform application programming interface OpenMP a method to train Semiparametric Support Vector Machines relying on Sparse Greedy Matrix Approximation (SGMA) and Iterated Re-Weighted Least Squares algorithm (IRWLS). We take advantage of the matrix formulation of SGMA and IRWLS. We recursively apply the partitioned matrix inversion lemma and other matrix decompositions to obtain a simple procedure to parallelize SVMS with good performance and computational efficiency.

[1]  Bernhard Schölkopf,et al.  Fast Approximation of Support Vector Kernel Expansions, and an Interpretation of Clustering as Approximation in Feature Spaces , 1998, DAGM-Symposium.

[2]  Bernhard Schölkopf,et al.  Improving the accuracy and speed of support vector learning machines , 1997, NIPS 1997.

[3]  Ángel Navia-Vázquez.  Support vector perceptrons , 2007, Neurocomputing.

[4]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[5]  William H. Press,et al.  Numerical recipes in C , 2002 .

[6]  S. Sathiya Keerthi,et al.  Parallel sequential minimal optimization for the training of support vector machines , 2006, IEEE Trans. Neural Networks.

[7]  Christoforos E. Kozyrakis,et al.  Evaluating MapReduce for Multi-core and Multiprocessor Systems , 2007, 2007 IEEE 13th International Symposium on High Performance Computer Architecture.

[8]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[9]  Samy Bengio,et al.  A Parallel Mixture of SVMs for Very Large Scale Problems , 2001, Neural Computation.

[10]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[11]  Ángel Navia-Vázquez,et al.  Compact multi-class support vector machine , 2007, Neurocomputing.

[12]  Aníbal R. Figueiras-Vidal,et al.  Growing support vector classifiers with controlled complexity , 2003, Pattern Recognit..

[13]  Emilio Parrado-Hernández,et al.  Distributed support vector machines , 2006, IEEE Trans. Neural Networks.

[14]  Ivor W. Tsang,et al.  The pre-image problem in kernel methods , 2003, IEEE Transactions on Neural Networks.

[15]  Fernando Pérez-Cruz,et al.  Fast Training of Support Vector Classifiers , 2000, NIPS.

[16]  Fernando Pérez-Cruz,et al.  Weighted least squares training of support vector classifiers leading to compact and adaptive schemes , 2001, IEEE Trans. Neural Networks.

[17]  Bernhard Schölkopf,et al.  Sparse Greedy Matrix Approximation for Machine Learning , 2000, International Conference on Machine Learning.

[18]  John R. Williams,et al.  Parallel multiclass classification using SVMs on GPUs , 2010, GPGPU-3.

[19]  Jian-xiong Dong,et al.  A Fast Parallel Optimization for Training Support Vector Machine , 2003, MLDM.