Branch and Bound for Semi-Supervised Support Vector Machines

Semi-supervised SVMs (S3VM) attempt to learn low-density separators by maximizing the margin over labeled and unlabeled examples. The associated optimization problem is non-convex. To examine the full potential of S3VMs modulo local minima problems in current implementations, we apply branch and bound techniques for obtaining exact, globally optimal solutions. Empirical evidence suggests that the globally optimal solution can return excellent generalization performance in situations where other implementations fail completely. While our current implementation is only applicable to small datasets, we discuss variants that can potentially lead to practically useful algorithms.

[1]  S. Sathiya Keerthi,et al.  Deterministic annealing for semi-supervised kernel machines , 2006, ICML.

[2]  Panos M. Pardalos,et al.  Computational aspects of a branch and bound algorithm for quadratic zero-one programming , 1990, Computing.

[3]  Jason Weston,et al.  Large Scale Transductive SVMs , 2006, J. Mach. Learn. Res..

[4]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[5]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[6]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[7]  Zoubin Ghahramani,et al.  Learning from labeled and unlabeled data with label propagation , 2002 .

[8]  Alexander Zien,et al.  A continuation method for semi-supervised SVMs , 2006, ICML.

[9]  Alexander Zien,et al.  Semi-Supervised Classification by Low Density Separation , 2005, AISTATS.

[10]  Matthias Seeger,et al.  A Taxonomy for Semi-Supervised Learning Methods , 2006, Semi-Supervised Learning.

[11]  W. Gander,et al.  A constrained eigenvalue problem , 1988 .

[12]  Alexander Zien,et al.  A Taxonomy for Semi-Supervised Learning Methods , 2006 .

[13]  Ayhan Demiriz,et al.  Semi-Supervised Support Vector Machines , 1998, NIPS.

[14]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[15]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[16]  Mikhail Belkin,et al.  Beyond the point cloud: from transductive to semi-supervised learning , 2005, ICML.