Vapnik-Chervonenkis bounds for generalization

The authors review the Vapnik and Chervonenkis theorem as applied to the problem of generalization. By combining some of the technical modifications proposed in the literature they derive tighter bounds and a new version of the theorem bounding the accuracy in the estimation of generalization probabilities from finite samples. A critical discussion and comparison with the results from statistical mechanics is given.

[1]  W. Hoeffding.  On the Distribution of the Number of Successes in Independent Trials , 1956 .

[2]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[3]  Norbert Sauer,et al.  On the Density of Families of Sets , 1972, J. Comb. Theory A.

[4]  L. Devroye.  Bounds for the Uniform Deviation of Empirical Measures , 1982 .

[5]  École d'été de probabilités de Saint-Flour,et al.  École d'Été de Probabilités de Saint-Flour XII - 1982 , 1984 .

[6]  David Haussler,et al.  Predicting (0, 1)-functions on randomly drawn points , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[7]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[8]  F. Vallet.  The Hebb Rule for Learning Linearly Separable Boolean Functions: Learning and Generalization , 1989 .

[9]  E. Gardner,et al.  Three unfinished works on the optimal storage capacity of networks , 1989 .

[10]  Yaser S. Abu-Mostafa,et al.  The Vapnik-Chervonenkis Dimension: Information versus Complexity in Learning , 1989, Neural Computation.

[11]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[12]  Györgyi,et al.  First-order transition to perfect generalization in a neural network with binary synapses. , 1990, Physical review. A, Atomic, molecular, and optical physics.

[13]  Van den Broeck C,et al.  Learning in feedforward Boolean networks. , 1990, Physical review. A, Atomic, molecular, and optical physics.

[14]  M. Opper,et al.  On the ability of the optimal perceptron to generalise , 1990 .

[15]  Sompolinsky,et al.  Learning from examples in large neural networks. , 1990, Physical review letters.

[16]  Vijay K. Samalam,et al.  Exhaustive Learning , 1990, Neural Computation.

[17]  Opper,et al.  Generalization performance of Bayes optimal classification algorithm for learning a perceptron. , 1991, Physical review letters.

[18]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[19]  Sompolinsky,et al.  Statistical mechanics of learning from examples. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[20]  Meir,et al.  Calculation of learning curves for inconsistent algorithms. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[21]  C. Van Den Broeck,et al.  Clipped-Hebbian Training of the Perceptron , 1993 .

[22]  T. Watkin,et al.  THE STATISTICAL-MECHANICS OF LEARNING A RULE , 1993 .