Stable Transductive Learning

We develop a new error bound for transductive learning algorithms. The slack term in the new bound is a function of a relaxed notion of transductive stability, which measures the sensitivity of the algorithm to most pairwise exchanges of training and test set points. Our bound is based on a novel concentration inequality for symmetric functions of permutations. We also present a simple sampling technique that can estimate, with high probability, the weak stability of transductive learning algorithms with respect to a given dataset. We demonstrate the usefulness of our estimation technique on a well known transductive learning algorithm.

[1]  T. Huang.  Performance Comparisons of Semi-Supervised Learning Algorithms , 2005 .

[2]  Mikhail Belkin,et al.  Regularization and Semi-supervised Learning on Large Graphs , 2004, COLT.

[3]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[4]  T. Poggio,et al.  STABILITY RESULTS IN LEARNING THEORY , 2005 .

[5]  S. Kutin.  Extensions to McDiarmid's inequality when dierences are bounded with high probability , 2002 .

[6]  M. Talagrand.  Majorizing measures: the generic chaining , 1996 .

[7]  André Elisseeff,et al.  Stability and Generalization , 2002, J. Mach. Learn. Res..

[8]  Partha Niyogi,et al.  Almost-everywhere Algorithmic Stability and Generalization Error , 2002, UAI.

[9]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[10]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[11]  中澤 真,et al.  Devroye, L., Gyorfi, L. and Lugosi, G. : A Probabilistic Theory of Pattern Recognition, Springer (1996). , 1997 .

[12]  Bernhard Schölkopf,et al.  Learning Theory and Kernel Machines , 2003, Lecture Notes in Computer Science.

[13]  Ran El-Yaniv,et al.  Explicit Learning Curves for Transduction and Application to Clustering and Compression Algorithms , 2004, J. Artif. Intell. Res..

[14]  Kazuoki Azuma.  WEIGHTED SUMS OF CERTAIN DEPENDENT RANDOM VARIABLES , 1967 .

[15]  Ali Esmaili,et al.  Probability and Random Processes , 2005, Technometrics.

[16]  T. Poggio,et al.  Statistical Learning: Stability is Sufficient for Generalization and Necessary and Sufficient for Consistency of Empirical Risk Minimization , 2002 .

[17]  John Langford,et al.  PAC-MDL Bounds , 2003, COLT.

[18]  V. Vapnik.  Estimation of Dependences Based on Empirical Data , 2006 .

[19]  R. A. Doney,et al.  4. Probability and Random Processes , 1993 .

[20]  Dana Ron,et al.  Algorithmic Stability and Sanity-Check Bounds for Leave-one-Out Cross-Validation , 1997, COLT.

[21]  Don R. Hush,et al.  Stability of Unstable Learning Algorithms , 2007, Machine Learning.

[22]  M. Ledoux.  The concentration of measure phenomenon , 2001 .

[23]  Bruce G. Lindsay,et al.  Approximate medians and other quantiles in one pass and with limited memory , 1998, SIGMOD '98.

[24]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.