An accelerator for support vector machines based on the local geometrical information and data partition

The support vector machines (SVM) is difficult to deal with large datasets for its low training efficiency. One of the important solutions has been developed by dividing a whole dataset into smaller subsets with data partition and combining the results of the classifiers over the divided subsets. However, traditional data partition approaches are difficult to preserve the class boundary of the dataset or control the size of divided subsets, so that their performance will be greatly influenced. To overcome this difficulty, we propose an accelerator for SVM algorithm based on the local geometrical information. In this algorithm, the feature space is divided into several regions with the approximately equal number of training instances by linear projection, and then each SVM classifier trained over the extended region only predicts the unlabeled instances within that original region. The proposed algorithm can not only hold the decision boundary of the raw data, but also saves a lot of execution time for implementing it in a parallel environment. Furthermore, the number of instances within each divided regions can be effectively controlled; it is conducive to choose the complexity of the execution in each of the processors. Experiments show that the classification performance of the proposed algorithm compares favorably with four state-of-the-art algorithms with the least training time.

[1]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[2]  Yuan-Hai Shao,et al.  Least squares recursive projection twin support vector machine for multi-class classification , 2015, International Journal of Machine Learning and Cybernetics.

[3]  Christian Cachin,et al.  Pedagogical pattern selection strategies , 1994, Neural Networks.

[4]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[5]  Koby Crammer,et al.  Trading representability for scalability: adaptive multi-hyperplane machine for nonlinear classification , 2011, KDD.

[6]  Xinjun Peng,et al.  A structural information-based twin-hypersphere support vector machine classifier , 2017, Int. J. Mach. Learn. Cybern..

[7]  Nicolás García-Pedrajas,et al.  Democratic instance selection: A linear complexity instance selection algorithm based on classifier ensemble concepts , 2010, Artif. Intell..

[8]  S. Sathiya Keerthi,et al.  Building Support Vector Machines with Reduced Classifier Complexity , 2006, J. Mach. Learn. Res..

[9]  Xian-Da Zhang,et al.  Matrix Analysis and Applications , 2017 .

[10]  Annabella Astorino,et al.  Scaling Up Support Vector Machines Using Nearest Neighbor Condensation , 2010, IEEE Transactions on Neural Networks.

[11]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[12]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[13]  Zhuang Wang,et al.  Scaling Up Kernel SVM on Limited Resources: A Low-Rank Linearization Approach , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Alexander J. Smola,et al.  Fastfood: Approximate Kernel Expansions in Loglinear Time , 2014, ArXiv.

[15]  Yu Xue,et al.  Weighted linear loss multiple birth support vector machine based on information granulation for multi-class classification , 2017, Pattern Recognit..

[16]  Giles M. Foody,et al.  The significance of border training patterns in classification by a feedforward neural network using back propagation learning , 1999 .

[17]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[18]  Samy Bengio,et al.  A Parallel Mixture of SVMs for Very Large Scale Problems , 2001, Neural Computation.

[19]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[20]  Senén Barro,et al.  Do we need hundreds of classifiers to solve real world classification problems? , 2014, J. Mach. Learn. Res..

[21]  Jieping Ye,et al.  Scaling SVM and Least Absolute Deviations via Exact Data Reduction , 2013, ICML.

[22]  Yuan-Hai Shao,et al.  Multiple recursive projection twin support vector machine for multi-class classification , 2014, International Journal of Machine Learning and Cybernetics.

[23]  François Poulet,et al.  Random Local SVMs for Classifying Large Datasets , 2015, FDSE.

[24]  Mauricio Kugler,et al.  CombNET-III: A Support Vector Machine Based Large Scale Classifier with Probabilistic Framework , 2006, IEICE Trans. Inf. Syst..

[25]  Shigeo Abe,et al.  Fusing sequential minimal optimization and Newton’s method for support vector training , 2014, International Journal of Machine Learning and Cybernetics.

[26]  Inderjit S. Dhillon,et al.  A Divide-and-Conquer Solver for Kernel Support Vector Machines , 2013, ICML.

[27]  Yuan-Hai Shao,et al.  Laplacian smooth twin support vector machine for semi-supervised classification , 2013, International Journal of Machine Learning and Cybernetics.

[28]  Aníbal R. Figueiras-Vidal,et al.  Sample selection via clustering to construct support vector-like classifiers , 1999, IEEE Trans. Neural Networks.

[29]  Xianli Pan,et al.  A safe screening based framework for support vector regression , 2018, Neurocomputing.

[30]  Francisco Herrera,et al.  Prototype Selection for Nearest Neighbor Classification: Taxonomy and Empirical Study , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[32]  C. Krishna Mohan,et al.  DiP-SVM : Distribution Preserving Kernel Support Vector Machine for Big Data , 2017, IEEE Transactions on Big Data.

[33]  Arie Ben-David,et al.  A lot of randomness is hiding in accuracy , 2007, Eng. Appl. Artif. Intell..

[34]  Prasoon Goyal,et al.  Local Deep Kernel Learning for Efficient Non-linear SVM Prediction , 2013, ICML.

[35]  Purnamrita Sarkar,et al.  A scalable bootstrap for massive data , 2011, 1112.5016.

[36]  Shifei Ding,et al.  An overview on semi-supervised support vector machine , 2017, Neural Computing and Applications.

[37]  Igor Durdanovic,et al.  Parallel Support Vector Machines: The Cascade SVM , 2004, NIPS.

[38]  Massimiliano Pontil,et al.  Properties of Support Vector Machines , 1998, Neural Computation.

[39]  R. Suganya,et al.  Data Mining Concepts and Techniques , 2010 .

[40]  Yuhua Li,et al.  Selecting Critical Patterns Based on Local Geometrical and Statistical Information , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Xianli Pan,et al.  Safe Screening Rules for Accelerating Twin Support Vector Machine Classification , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Daniel Asimov,et al.  The grand tour: a tool for viewing multidimensional data , 1985 .

[43]  S. Halgamuge,et al.  Reducing the Number of Training Samples for Fast Support Vector Machine Classification , 2004 .

[44]  Sungzoon Cho,et al.  Neighborhood PropertyBased Pattern Selection for Support Vector Machines , 2007, Neural Computation.

[45]  Gary Doran,et al.  A theoretical and empirical analysis of support vector machine methods for multiple-instance classification , 2014, Machine Learning.