Selecting Informative Universum Sample for Semi-Supervised Learning

The Universum sample, which is defined as the sample that doesn't belong to any of the classes the learning task concerns, has been proved to be helpful in both supervised and semi-supervised settings. The former works treat the Universum samples equally. Our research found that not all the Universum samples are helpful, and we propose a method to pick the informative ones, i.e., in-between Universum samples. We also set up a new semi-supervised framework to incorporate the in-between Universum samples. Empirical experiments show that our method outperforms the former ones.

[1]  Mikhail Belkin,et al.  Regularization and Semi-supervised Learning on Large Graphs , 2004, COLT.

[2]  E. L. Lawler,et al.  Branch-and-Bound Methods: A Survey , 1966, Oper. Res..

[3]  C. McDiarmid,et al.  Probabilistic analysis of tree search , 1990 .

[4]  L. Rosasco,et al.  Manifold Regularization , 2007 .

[5]  Alexander Zien,et al.  Transductive Inference and Semi-Supervised Learning , 2006 .

[6]  Leonard M. Freeman,et al.  A set of measures of centrality based upon betweenness , 1977 .

[7]  L. Freeman.  Centrality in social networks conceptual clarification , 1978 .

[8]  Vladimir Vapnik.  Transductive Inference and Semi-Supervised Learning , 2006, Semi-Supervised Learning.

[9]  Peter C. Cheeseman,et al.  Where the Really Hard Problems Are , 1991, IJCAI.

[10]  Don Coppersmith,et al.  Constructive Bounds and Exact Expectations for the Random Assignment Problem , 1998, RANDOM.

[11]  Hector J. Levesque,et al.  Hard and Easy Distributions of SAT Problems , 1992, AAAI.

[12]  Weixiong Zhang,et al.  Phase Transitions and Backbones of 3-SAT and Maximum 3-SAT , 2001, CP.

[13]  P. Toth,et al.  Some New Branching and Bounding Criteria for the Asymmetric Travelling Salesman Problem , 1980 .

[14]  Paolo Toth,et al.  Linear Assignment Problems , 1987 .

[15]  William W. Cohen,et al.  Proceedings of the 23rd international conference on Machine learning , 2006, ICML 2008.

[16]  Marc Toussaint,et al.  Probabilistic inference for solving discrete and continuous state Markov Decision Processes , 2006, ICML.

[17]  Qiang Yang,et al.  Self-taught clustering , 2008, ICML '08.

[18]  Richard M. Karp,et al.  Searching for an Optimal Path in a Tree with Random Costs , 1983, Artif. Intell..

[19]  Rajat Raina,et al.  Self-taught learning: transfer learning from unlabeled data , 2007, ICML '07.

[20]  Bernhard Schölkopf,et al.  An Analysis of Inference with the Universum , 2007, NIPS.

[21]  Weixiong Zhang,et al.  A Study of Complexity Transitions on the Asymmetric Traveling Salesman Problem , 1996, Artif. Intell..

[22]  Alan M. Frieze,et al.  When is the Assignment Bound Tight for the Asymmetric Traveling Salesman Problem? , 1992, IPCO.

[23]  ATSPDavid S. JohnsonAT.  Experimental Analysis of Heuristics for the Stsp , 2001 .

[24]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[25]  Christian Borgs,et al.  Phase transition and finite‐size scaling for the integer partitioning problem , 2001, Random Struct. Algorithms.

[26]  Toby Walsh,et al.  The TSP Phase Transition , 1996, Artif. Intell..

[27]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[28]  M. Henkel.  Finite-Size Scaling , 1999 .

[29]  S. S. Sengupta,et al.  The traveling salesman problem , 1961 .

[30]  Jason Weston,et al.  Inference with the Universum , 2006, ICML.

[31]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[32]  Toby Walsh,et al.  Backbones in Optimization and Approximation , 2001, IJCAI.

[33]  Vikas Sindhwani,et al.  On Manifold Regularization , 2005, AISTATS.

[34]  Richard E. Korf,et al.  Performance of Linear-Space Search Algorithms , 1995, Artif. Intell..

[35]  S. Kirkpatrick,et al.  Configuration space analysis of travelling salesman problems , 1985 .

[36]  Abraham P. Punnen,et al.  The traveling salesman problem and its variations , 2007 .

[37]  Fei Wang,et al.  Semi-Supervised Classification with Universum , 2008, SDM.

[38]  Mandell Bellmore,et al.  Pathology of Traveling-Salesman Subtour-Elimination Algorithms , 1971, Oper. Res..

[39]  Paolo Toth,et al.  Exact solution of large-scale, asymmetric traveling salesman problems , 1995, TOMS.

[40]  Thorsten Joachims,et al.  Transductive Learning via Spectral Graph Partitioning , 2003, ICML.

[41]  Rémi Monasson,et al.  Determining computational complexity from characteristic ‘phase transitions’ , 1999, Nature.

[42]  Alan M. Frieze,et al.  The probabilistic relationship between the assignment and asymmetric traveling salesman problems , 2001, SODA '01.