Canonical duality solution for alternating support vector machine

Support vector machine (SVM) is one of the most popular machine learning methods and is educed from a binary data classification problem. In this paper, the canonical duality theory is used to solve the normal model of SVM. Several examples are illustrated to show that the exact solution can be obtained after the canonical duality problem being solved. Moreover, the support vectors can be located by non-zero elements of the canonical dual solution.

[1]  K. Schittkowski.  Optimal parameter selection in support vector machines , 2005 .

[2]  Yuh-Jye Lee,et al.  epsilon-SSVR: A Smooth Support Vector Machine for epsilon-Insensitive Regression , 2005, IEEE Trans. Knowl. Data Eng..

[3]  J. Platt.  Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[4]  Jorge Nocedal,et al.  On the Behavior of the Gradient Norm in the Steepest Descent Method , 2002, Comput. Optim. Appl..

[5]  Ya-Xiang Yuan,et al.  A modified BFGS algorithm for unconstrained optimization , 1991 .

[6]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[7]  Patrick T. Harker,et al.  Smooth Approximations to Nonlinear Complementarity Problems , 1997, SIAM J. Optim..

[8]  Y. Ye,et al.  On Homotopy-Smoothing Methods for Variational Inequalities , 1999 .

[9]  David Yang Gao,et al.  Canonical Dual Transformation Method and Generalized Triality Theory in Nonsmooth Global Optimization , 2000, J. Glob. Optim..

[10]  Yuh-Jye Lee,et al.  SSVM: A Smooth Support Vector Machine for Classification , 2001, Comput. Optim. Appl..

[11]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[12]  Olvi L. Mangasarian,et al.  Smoothing methods for convex inequalities and linear complementarity problems , 1995, Math. Program..

[13]  Gary William Flake,et al.  Efficient SVM Regression Training with SMO , 2002, Machine Learning.

[14]  David Yang Gao.  Complete Solutions and Extremality Criteria to Polynomial Optimization Problems , 2006, J. Glob. Optim..

[15]  D. Gao.  Perfect duality theory and complete solutions to a class of global optimization problems , 2003 .

[16]  Pavel Laskov,et al.  Feasible Direction Decomposition Algorithms for Training Support Vector Machines , 2002, Machine Learning.

[17]  Matteo Rocca,et al.  Two approaches toward constrained vector optimization and identity of the solutions , 2005 .

[18]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[19]  Chih-Jen Lin,et al.  A Simple Decomposition Method for Support Vector Machines , 2002, Machine Learning.

[20]  YuBo Yuan,et al.  A Polynomial Smooth Support Vector Machine for Classification , 2005, ADMA.

[21]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[22]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[23]  S. Sathiya Keerthi,et al.  A Fast Dual Algorithm for Kernel Logistic Regression , 2002, 2007 International Joint Conference on Neural Networks.

[24]  Xiaojun Chen,et al.  Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities , 1998, Math. Comput..

[25]  Ping-Feng Pai,et al.  System reliability forecasting by support vector machines with genetic algorithms , 2006, Math. Comput. Model..

[26]  K. L. Mak,et al.  Airfoil design via optimal control theory , 2005 .

[27]  Glenn Fung,et al.  Finite Newton method for Lagrangian support vector machine classification , 2003, Neurocomputing.

[28]  Bernhard Schölkopf,et al.  Support vector learning , 1997 .

[29]  David R. Musicant,et al.  Successive overrelaxation for support vector machines , 1999, IEEE Trans. Neural Networks.

[30]  Yuan Yu.  Polynomial Smooth Support Vector Machine(PSSVM) , 2005 .

[31]  Zhongzhi Shi,et al.  A novel classification method based on hypersurface , 2003 .

[32]  Thorsten Joachims,et al.  Making large-scale support vector machine learning practical , 1999 .

[33]  David Yang Gao,et al.  Sufficient conditions and perfect duality in nonconvex minimization with inequality constraints , 2005 .

[34]  David Yang Gao,et al.  Canonical Duality Theory and Solutions to Constrained Nonconvex Quadratic Programming , 2004, J. Glob. Optim..

[35]  Olvi L. Mangasarian,et al.  A class of smoothing functions for nonlinear and mixed complementarity problems , 1996, Comput. Optim. Appl..

[36]  Edward Y. Chang,et al.  KDX: an indexer for support vector machines , 2006, IEEE Transactions on Knowledge and Data Engineering.

[37]  Alireza Ghaffari Hadigheh,et al.  Generalized support set invariancy sensitivity analysis in linear optimization , 2006 .

[38]  Y. Ye,et al.  On Homotopy-Smoothing Methods for Box-Constrained Variational Inequalities , 1999 .

[39]  Zhiyou Wu,et al.  Quadratic smoothing approximation to $l_1$ exact penalty function in global optimization , 2005 .

[40]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[41]  Andreu Català,et al.  Dual unification of bi-class support vector machine formulations , 2006, Pattern Recognit..