General bounds on statistical query learning and PAC learning with noise via hypothesis boosting

We derive general bounds on the complexity of learning in the statistical query model and in the PAC model with classification noise. We do so by considering the problem of boosting the accuracy of weak learning algorithms which fall within the statistical query model. This new model was introduced by M. Kearns (1993) to provide a general framework for efficient PAC learning in the presence of classification noise.<<ETX>>

[1]  Scott E. Decatur.  Statistical queries and faulty PAC oracles , 1993, COLT '93.

[2]  Javed A. Aslam,et al.  Specification and simulation of statistical query algorithms for efficiency and noise tolerance , 1995, COLT '95.

[3]  Robert E. Schapire,et al.  The strength of weak learnability , 1990, Mach. Learn..

[4]  榊原 康文,et al.  Algorithmic learning of formal languages and decision trees , 1991 .

[5]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[6]  Robert E. Schapire,et al.  On the sample complexity of weak learning , 1990, COLT '90.

[7]  Dana Angluin,et al.  Computational learning theory: survey and selected bibliography , 1992, STOC '92.

[8]  Harris Drucker,et al.  Improving Performance in Neural Networks Using a Boosting Algorithm , 1992, NIPS.

[9]  B. Harshbarger.  An Introduction to Probability Theory and its Applications, Volume I , 1958 .

[10]  Manfred K. Warmuth,et al.  Learning integer lattices , 1990, COLT '90.

[11]  P. Laird.  Learning from Good and Bad Data , 1988 .

[12]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[13]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[14]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[15]  Leslie G. Valiant,et al.  A general lower bound on the number of examples needed for learning , 1988, COLT '88.

[16]  Ronald L. Graham,et al.  Concrete mathematics - a foundation for computer science , 1991 .

[17]  Bill Broyles.  Notes , 1907, The Classical Review.

[18]  Yoav Freund,et al.  An improved boosting algorithm and its implications on learning complexity , 1992, COLT '92.

[19]  Robert E. Schapire,et al.  Design and analysis of efficient learning algorithms , 1992, ACM Doctoral dissertation award ; 1991.

[20]  Hans Ulrich Simon,et al.  General bounds on the number of examples needed for learning probabilistic concepts , 1993, COLT '93.

[21]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1990, COLT '90.

[22]  Javed A. Aslam,et al.  Improved Noise-Tolerant Learning and Generalized Statistical Queries , 1994 .

[23]  Norbert Sauer,et al.  On the Density of Families of Sets , 1972, J. Comb. Theory A.

[24]  Michael Kearns,et al.  Efficient noise-tolerant learning from statistical queries , 1993, STOC.