论文信息 - Generalization Bounds for Linear Learning Algorithms

Generalization Bounds for Linear Learning Algorithms

We study generalization properties of linear learning algo rithms and develop a data dependent approach that is used to derive genera lization bounds that depend on the margin distribution. Our method ma kes use of random projection techniques to allow the use of existing VC dimension bounds in the effective, lower, dimension of the data. C omparisons with existing generalization bound show that our bounds are tighter and meaningful in cases existing bounds are not. Our methods is a l o shown to be useful as an analysis tool, alternative to learning cur ves.

Sariel Har-Peled | D. Roth | A. Garg

[1] W. B. Johnson,et al. Extensions of Lipschitz mappings into Hilbert space , 1984 .

[2] Nello Cristianini,et al. An introduction to Support Vector Machines , 2000 .

[3] Manfred K. Warmuth,et al. The Perceptron Algorithm Versus Winnow: Linear Versus Logarithmic Mistake Bounds when Few Input Variables are Relevant (Technical Note) , 1997, Artif. Intell..

[4] John Shawe-Taylor,et al. Classification Accuracy Based on Observed Margin , 1998, Algorithmica.

[5] Dan Roth,et al. Learning to Resolve Natural Language Ambiguities: A Unified Approach , 1998, AAAI/IAAI.

[6] Dan Roth,et al. Learning Coherent Concepts , 2001, ALT.

[7] Nello Cristianini,et al. Further results on the margin distribution , 1999, COLT '99.

[8] Dana Ron,et al. An experimental and theoretical comparison of model selection methods , 1995, COLT '95.

[9] David Haussler,et al. Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[10] Robert E. Schapire,et al. Efficient distribution-free learning of probabilistic concepts , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[11] Manfred K. Warmuth,et al. The perceptron algorithm vs. Winnow: linear vs. logarithmic mistake bounds when few input variables are relevant , 1995, COLT '95.