On the Intrinsic Complexity of Learning

A new view of learning is presented. The basis of this view is a natural notion of reduction. We prove completeness and relative difficulty results. An infinite hierarchy of intrinsically more and more difficult to learn concepts is presented. Our results indicate that the complexity notion captured by our new notion of reduction differs dramatically from the traditional studies of the complexity of the algorithms performing learning tasks.

[1]  Rusins Freivalds,et al.  Inductive Inference of Recursive Functions: Qualitative Theory , 1991, Baltic Computer Science.

[2]  Hartley Rogers,et al.  Gödel numberings of partial recursive functions , 1958, Journal of Symbolic Logic.

[3]  D. Angluin.  Queries and Concept Learning , 1988 .

[4]  Rusins Freivalds.  Inductive Inference of Minimal Programs , 1990, COLT.

[5]  Manfred K. Warmuth.  Towards Representation Independence in PAC Learning , 1989, AII.

[6]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[7]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[8]  Paul Young,et al.  An introduction to the general theory of algorithms , 1978 .

[9]  Leonard Pitt,et al.  Reductions among prediction problems: on the difficulty of predicting automata , 1988, [1988] Proceedings. Structure in Complexity Theory Third Annual Conference.

[10]  Carl H. Smith,et al.  On the impact of forgetting on learning machines , 1993, COLT '93.

[11]  William I. Gasarch,et al.  Learning via queries to an oracle , 1989, COLT '89.

[12]  Carl H. Smith,et al.  On the impact of forgetting on learning machines , 1995, JACM.

[13]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[14]  Arun Sharma,et al.  On the intrinsic complexity of language identification , 1994, COLT '94.

[15]  Carl H. Smith,et al.  A Recursive Introduction to the Theory of Computation , 1994, Graduate Texts in Computer Science.

[16]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[17]  Arun Sharma,et al.  The Structure of Intrinsic Complexity of Learning , 1997, J. Symb. Log..

[18]  Carl H. Smith,et al.  On the role of procrastination for machine learning , 1992, COLT '92.

[19]  Stuart A. Kurtz,et al.  Degrees of inferability , 1992, COLT '92.

[20]  Carl H. Smith,et al.  On the Role of Procrastination in Machine Learning , 1993, Inf. Comput..