DCA Based Algorithms for Feature Selection in Semi-supervised Support Vector Machines

In this paper, we develop an efficient method for feature selection in Semi-Supervised Support Vector Machine (S3VM). Using an appropriate continuous approximation of the l0−norm, we reformulate the feature selection S3VM problem as a DC (Difference of Convex functions) program. DCA (DC Algorithm), an innovative approach in nonconvex programming is then developed to solve the resulting problem. Computational experiments on several real-world datasets show the efficiency and the scalability of our method.

[1]  Gabriele Steidl,et al.  Combined SVM-Based Feature Selection and Classification , 2005, Machine Learning.

[2]  Edoardo Amaldi,et al.  On the Approximability of Minimizing Nonzero Variables or Unsatisfied Relations in Linear Systems , 1998, Theor. Comput. Sci..

[3]  Ayhan Demiriz,et al.  Semi-Supervised Support Vector Machines , 1998, NIPS.

[4]  Le Thi Hoai An,et al.  A new efficient algorithm based on DC programming and DCA for clustering , 2007, J. Glob. Optim..

[5]  Le Thi Hoai An,et al.  A DC programming approach for feature selection in support vector machines learning , 2008, Adv. Data Anal. Classif..

[6]  Jason Weston,et al.  Large Scale Transductive SVMs , 2006, J. Mach. Learn. Res..

[7]  Alain Rakotomamonjy,et al.  Variable Selection Using SVM-based Criteria , 2003, J. Mach. Learn. Res..

[8]  Le Thi Hoai An,et al.  DC Programming Approach for a Class of Nonconvex Programs Involving l0 Norm , 2008, MCO.

[9]  Yufeng Liu,et al.  Multicategory ψ-Learning and Support Vector Machine: Computational Tools , 2005 .

[10]  O. Mangasarian,et al.  Semi-Supervised Support Vector Machines for Unlabeled Data Classification , 2001 .

[11]  Yoram Singer,et al.  Leveraging the margin more carefully , 2004, ICML.

[12]  Alan L. Yuille,et al.  The Concave-Convex Procedure , 2003, Neural Computation.

[13]  S. Sathiya Keerthi,et al.  Deterministic annealing for semi-supervised kernel machines , 2006, ICML.

[14]  Le Thi Hoai An,et al.  Optimization based DC programming and DCA for hierarchical clustering , 2007, Eur. J. Oper. Res..

[15]  S. Sathiya Keerthi,et al.  Optimization Techniques for Semi-Supervised Support Vector Machines , 2008, J. Mach. Learn. Res..

[16]  S. Sathiya Keerthi,et al.  Branch and Bound for Semi-Supervised Support Vector Machines , 2006, NIPS.

[17]  Jason Weston,et al.  Trading convexity for scalability , 2006, ICML.

[18]  Le Thi Hoai An,et al.  Solving a Class of Linearly Constrained Indefinite Quadratic Problems by D.C. Algorithms , 1997, J. Glob. Optim..

[19]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[20]  Le Thi Hoai An,et al.  The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Nonconvex Optimization Problems , 2005, Ann. Oper. Res..

[21]  Alexander Zien,et al.  Semi-Supervised Classification by Low Density Separation , 2005, AISTATS.

[22]  Paul S. Bradley,et al.  Feature Selection via Concave Minimization and Support Vector Machines , 1998, ICML.