Support Vector Machine

Support Vector Machine is one of the classical machine learning techniques that can still help solve big data classification problems. Especially, it can help the multidomain applications in a big data environment. However, the support vector machine is mathematically complex and computationally expensive. The main objective of this chapter is to simplify this approach using process diagrams and data flow diagrams to help readers understand theory and implement it successfully. To achieve this objective, the chapter is divided into three parts: (1) modeling of a linear support vector machine; (2) modeling of a nonlinear support vector machine; and (3) Lagrangian support vector machine algorithm and its implementations. The Lagrangian support vector machine with simple examples is also implemented using the R programming platform on Hadoop and non-Hadoop systems.

[1]  Jing Bu,et al.  Mean Shifts Identification Model in Bivariate Process Based on LS-SVM Pattern Recognizer , 2010, J. Digit. Content Technol. its Appl..

[2]  Alain Biem,et al.  Semisupervised Learning Using Bayesian Interpretation: Application to LS-SVM , 2011, IEEE Transactions on Neural Networks.

[3]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[4]  Guoyin Li,et al.  Support vector machine classifiers with uncertain knowledge sets via robust optimization , 2014 .

[5]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[6]  Md. Monirul Islam,et al.  A review on automatic image annotation techniques , 2012, Pattern Recognit..

[7]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[8]  Václav Hlavác,et al.  Multi-class support vector machine , 2002, Object recognition supported by user interaction for service robots.

[9]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[10]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[11]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[12]  Sungwan Bang,et al.  Adaptive sup-norm regularized simultaneous multiple quantiles regression , 2014 .

[13]  Huafu Chen,et al.  Two-class support vector data description , 2011, Pattern Recognit..

[14]  Mohamed Cheriet,et al.  New Formulation of SVM for Model Selection , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[15]  Ching Y. Suen,et al.  Automatic model selection for the optimization of SVM kernels , 2005, Pattern Recognit..

[16]  Gunnar Rätsch,et al.  Input space versus feature space in kernel-based methods , 1999, IEEE Trans. Neural Networks.

[17]  P. Zhao,et al.  The composite absolute penalties family for grouped and hierarchical variable selection , 2009, 0909.0411.

[18]  Jerzy W. Grzymala-Busse,et al.  Rough Sets , 1995, Commun. ACM.

[19]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[20]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[21]  G. Wahba,et al.  A NOTE ON THE LASSO AND RELATED PROCEDURES IN MODEL SELECTION , 2006 .

[22]  Zhengjun Cheng,et al.  Classification Models of Estrogen Receptor-B Ligands Based on PSO-Adaboost-SVM , 2010, J. Convergence Inf. Technol..

[23]  Hansheng Wang,et al.  Computational Statistics and Data Analysis a Note on Adaptive Group Lasso , 2022 .

[24]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[25]  Francesca Odone,et al.  Histogram intersection kernel for image classification , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[26]  Isabelle Guyon,et al.  Writer-adaptation for on-line handwritten character recognition , 1993, Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR '93).

[27]  Sungwan Bang,et al.  Simultaneous estimation and factor selection in quantile regression via adaptive sup-norm regularization , 2012, Comput. Stat. Data Anal..

[28]  Ji Zhu,et al.  Variable selection for multicategory SVM via sup-norm regularization , 2006 .

[29]  Marti A. Hearst.  Trends & Controversies: Support Vector Machines , 1998, IEEE Intell. Syst..

[30]  Thorsten Joachims,et al.  Making large-scale support vector machine learning practical , 1999 .

[31]  Vaithilingam Jeyakumar,et al.  Simultaneous classification and feature selection via convex quadratic programming with application to HIV-associated neurocognitive disorder assessment , 2010, Eur. J. Oper. Res..

[32]  Gavin C. Cawley,et al.  Fast exact leave-one-out cross-validation of sparse least-squares support vector machines , 2004, Neural Networks.

[33]  Robert Tibshirani,et al.  1-norm Support Vector Machines , 2003, NIPS.

[34]  Zdzislaw Pawlak,et al.  Rough sets and intelligent data analysis , 2002, Inf. Sci..

[35]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[36]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[37]  Olivier Chapelle,et al.  Model Selection for Support Vector Machines , 1999, NIPS.

[38]  Mohamed Cheriet,et al.  Optimizing resources in model selection for support vector machines , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[39]  David R. Musicant,et al.  Active Support Vector Machine Classification , 2000, NIPS.

[40]  Yuan Yao,et al.  Combining flat and structured representations for fingerprint classification with recursive neural networks and support vector machines , 2003, Pattern Recognit..

[41]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[42]  A. E. Hoerl,et al.  Ridge regression: biased estimation for nonorthogonal problems , 2000 .

[43]  Peter J. Rousseeuw,et al.  Finding Groups in Data: An Introduction to Cluster Analysis , 1990 .

[44]  Hui Zou.  An Improved 1-norm SVM for Simultaneous Classification and Variable Selection , 2007, AISTATS.

[45]  Mohamed Cheriet,et al.  Model selection for the LS-SVM. Application to handwriting recognition , 2009, Pattern Recognit..