Algebraic multigrid support vector machines

The support vector machine is a flexible optimization-based technique widely used for classification problems. In practice, its training part becomes computationally expensive on large-scale data sets because of such reasons as the complexity and number of iterations in parameter fitting methods, underlying optimization solvers, and nonlinearity of kernels. We introduce a fast multilevel framework for solving support vector machine models that is inspired by the algebraic multigrid. Significant improvement in the running has been achieved without any loss in the quality. The proposed technique is highly beneficial on imbalanced sets. We demonstrate computational results on publicly available and industrial data sets.

[1]  Federico Girosi,et al.  An improved training algorithm for support vector machines , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[2]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[3]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[4]  Hyun-Chul Kim,et al.  Constructing support vector machine ensemble , 2003, Pattern Recognit..

[5]  D. Ron,et al.  Multigrid Solvers and Multilevel Optimization Strategies , 2003 .

[6]  Igor Durdanovic,et al.  Parallel Support Vector Machines: The Cascade SVM , 2004, NIPS.

[7]  Ding-Xuan Zhou,et al.  SVM Soft Margin Classifiers: Linear Programming versus Quadratic Programming , 2005, Neural Computation.

[8]  Chih-Jen Lin,et al.  Working Set Selection Using Second Order Information for Training Support Vector Machines , 2005, J. Mach. Learn. Res..

[9]  Thomas A. Manteuffel,et al.  An energy‐based AMG coarsening strategy , 2006, Numer. Linear Algebra Appl..

[10]  Ilya Safro,et al.  Graph minimum linear arrangement by multilevel weighted edge contractions , 2006, J. Algorithms.

[11]  Achi Brandt,et al.  Fast multiscale clustering and manifold identification , 2006, Pattern Recognit..

[12]  Ronen Basri,et al.  Hierarchy and adaptivity in segmenting visual scenes , 2006, Nature.

[13]  Inderjit S. Dhillon,et al.  Weighted Graph Cuts without Eigenvectors A Multilevel Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Edward Y. Chang,et al.  Parallelizing Support Vector Machines on Distributed Computers , 2007, NIPS.

[15]  Su-Yun Huang,et al.  Model selection for support vector machines via uniform design , 2007, Comput. Stat. Data Anal..

[16]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[17]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[18]  Andreas Noack,et al.  Multi-level Algorithms for Modularity Clustering , 2008, SEA.

[19]  Yousef Saad,et al.  Multilevel manifold learning with application to spectral clustering , 2010, CIKM.

[20]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[21]  Ilya Safro,et al.  Relaxation-based coarsening and multiscale graph organization , 2010, Multiscale Model. Simul..

[22]  Ilya Safro,et al.  Multiscale approach for the network compression-friendly ordering , 2010, J. Discrete Algorithms.

[23]  Ilya Safro,et al.  Algebraic Distance on Graphs , 2011, SIAM J. Sci. Comput..

[24]  Giorgio Vittadini,et al.  Multilevel dimensionality-reduction methods , 2013, Stat. Methods Appl..

[25]  Sven Leyffer,et al.  Fast response to infection spread and cyber attacks on large-scale networks , 2012, J. Complex Networks.

[26]  Peter Sanders,et al.  Advanced Coarsening Schemes for Graph Partitioning , 2012, ACM J. Exp. Algorithmics.

[27]  Johan A. K. Suykens,et al.  EnsembleSVM: a library for ensemble learning using support vector machines , 2014, J. Mach. Learn. Res..

[28]  Ilya Safro,et al.  Scalable Multilevel Support Vector Machines , 2015, ICCS.

[29]  Francisco Herrera,et al.  Cost-sensitive linguistic fuzzy rule based classification systems under the MapReduce framework for imbalanced big data , 2015, Fuzzy Sets Syst..

[30]  Shuaiwen Song,et al.  Scaling Support Vector Machines on modern HPC platforms , 2015, J. Parallel Distributed Comput..

[31]  William Gropp,et al.  PETSc Users Manual Revision 3.4 , 2016 .

[32]  Ilya Safro,et al.  Multilevel Weighted Support Vector Machine for Classification on Healthcare Data with Missing Values , 2016, PloS one.