Quantifying the Inductive Bias in Concept Learning (Extended Abstract)

We show that the notion of bias in inductive concept learning can be quantified in a way that directly relates to learning performance, and that this quantitative theory of bias can provide guidance in the design of effective learning algorithms. We apply this idea by measuring some common language biases, including restriction to conjunctive concepts and conjunctive concepts with internal disjunction, and, guided by these measurements, develop learning algorithms or these classes of concepts that have provably good convergence properties.

[1]  David S. Johnson,et al.  Approximation algorithms for combinatorial problems , 1973, STOC.

[2]  Paul E. Utgoff,et al.  Shift of bias for inductive concept learning , 1984 .

[3]  Leslie G. Valiant,et al.  Learning Disjunction of Conjunctions , 1985, IJCAI.

[4]  David Haussler,et al.  Classifying learnable geometric concepts with the Vapnik-Chervonenkis dimension , 1986, STOC '86.

[5]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[6]  P. Assouad.  Densité et dimension , 1983 .

[7]  Richard M. Dudley,et al.  Some special vapnik-chervonenkis classes , 1981, Discret. Math..

[8]  Ranan B. Banerji,et al.  The Logic of Learning: A Basis for Pattern Recognition and for Improvement of Performance , 1985, Adv. Comput..

[9]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[10]  Ryszard S. Michalski,et al.  A Theory and Methodology of Inductive Learning , 1983, Artificial Intelligence.

[11]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[12]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[13]  Tom M. Mitchell,et al.  Generalization as Search , 2002 .

[14]  Alan Bundy,et al.  An Analytical Comparison of Some Rule-Learning Programs , 1985, Artif. Intell..

[15]  Judea Pearl,et al.  ON THE CONNECTION BETWEEN THE COMPLEXITY AND CREDIBILITY OF INFERRED MODELS , 1978 .