Robust Unsupervised and Semisupervised Bounded C-Support Vector Machines

Support Vector Machines (SVMs) have been dominant learning techniques for almost ten years, and mostly ap- plied to supervised learning problems. Recently two-class unsupervised and semi-supervised classification algorithms based on Bounded C-SVMs, Bounded -SVMs and La- grangian SVMs (LSVMs) respectively, which are relaxed to Semi-definite Programming (SDP), get good classifica- tion results. These support vector methods implicitly as- sume that training data in the optimization problems to be known exactly. But in practice, the training data are usually subjected to measurement noise. In this paper we proposed robust version to unsupervised and semi-supervised classifi- cation problems based on Bounded C-Support Vector Ma- chines, which trained by convex relaxation of the training criterion: find a labeling that yield a maximum margin on the training data with perturbations. But the problem has difficulty to compute, we will find its semi-definite relaxation that can approximate it well. Numerical results confirm the robustness of the proposed method. Keywords: Bounded Support Vector Machines, Semi- definite Programming, unsupervised learning, semi- supervised learning, robust

[1]  Bernhard Schölkopf,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[2]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[3]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[4]  Allen L. Soyster,et al.  Technical Note - Convex Programming with Set-Inclusive Constraints and Applications to Inexact Linear Programming , 1973, Oper. Res..

[5]  Melvyn Sim,et al.  The Price of Robustness , 2004, Oper. Res..

[6]  A Ben Tal,et al.  ROBUST SOLUTIONS TO UNCERTAIN PROGRAMS , 1999 .

[7]  Jie Wang,et al.  Knowledge and Information Systems REGULAR PAPER , 2006 .

[8]  Arkadi Nemirovski,et al.  Robust Convex Optimization , 1998, Math. Oper. Res..

[9]  Laurent El Ghaoui,et al.  Robust Solutions to Least-Squares Problems with Uncertain Data , 1997, SIAM J. Matrix Anal. Appl..

[10]  David R. Musicant,et al.  Successive overrelaxation for support vector machines , 1999, IEEE Trans. Neural Networks.

[11]  Jie Wang,et al.  NNMF-Based Factorization Techniques for High-Accuracy Privacy Protection on Non-negative-valued Datasets , 2006, Sixth IEEE International Conference on Data Mining - Workshops (ICDMW'06).

[12]  Arkadi Nemirovski,et al.  Robust solutions of uncertain linear programs , 1999, Oper. Res. Lett..

[13]  Dale Schuurmans,et al.  Maximum Margin Clustering , 2004, NIPS.

[14]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[15]  Laurent El Ghaoui,et al.  Robust Solutions to Uncertain Semidefinite Programs , 1998, SIAM J. Optim..

[16]  Elisa Bertino,et al.  Association rule hiding , 2004, IEEE Transactions on Knowledge and Data Engineering.

[17]  Arkadi Nemirovski,et al.  Robust solutions of Linear Programming problems contaminated with uncertain data , 2000, Math. Program..

[18]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[19]  Nello Cristianini,et al.  Convex Methods for Transduction , 2003, NIPS.

[20]  Kun Zhao,et al.  Unsupervised and Semi-supervised Lagrangian Support Vector Machines with Polyhedral Perturbations , 2007, 2009 Third International Symposium on Intelligent Information Technology Application.

[21]  L El Ghaoui,et al.  ROBUST SOLUTIONS TO LEAST-SQUARE PROBLEMS TO UNCERTAIN DATA MATRICES , 1997 .

[22]  John A. Hartigan,et al.  Clustering Algorithms , 1975 .