Ignoring data may be the only way to learn efficiently

Abstract In designing learning algorithms it seems quite reasonable to construct them in a way such that all data the algorithm already has obtained are correctly and completely reflected in the hypothesis the algorithm outputs on these data. However, this approach may totally fail, i.e. it may lead to the unsolvability of the learning problem, or it may exclude any efficient solution of it. In particular, we present a natural learning problem and prove that it can be solved in polynomial time if and only if the algorithm is allowed to ignore data.

[1]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part I , 1964, Inf. Control..

[2]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[3]  E. Mark Gold,et al.  Limiting recursion , 1965, Journal of Symbolic Logic.

[4]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[5]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[6]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[7]  Rolf Wiehagen.  Limes-Erkennung rekursiver Funktionen durch spezielle Strategien , 1975, J. Inf. Process. Cybern..

[8]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[9]  Rolf Wiehagen,et al.  Characterization Problems in the Theory of Inductive Inference , 1978, ICALP.

[10]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[11]  Dana Angluin,et al.  Finding Patterns Common to a Set of Strings , 1980, J. Comput. Syst. Sci..

[12]  Klaus P. Jantke,et al.  Combining Postulates of Naturalness in Inductive Inference , 1981, J. Inf. Process. Cybern..

[13]  Takeshi Shinohara,et al.  Polynomial Time Inference of Extended Regular Pattern Languages , 1983, RIMS Symposium on Software Science and Engineering.

[14]  Robert Peter Nix.  Editing by example (text) , 1983 .

[15]  Thomas Zeugmann,et al.  A-posteriori Characterizations in Inductive Inference of Recursive Functions , 1983, J. Inf. Process. Cybern..

[16]  Carl H. Smith,et al.  Inductive Inference: Theory and Methods , 1983, CSUR.

[17]  Robert Nix,et al.  Editing by example , 1985, POPL '84.

[18]  Mark A. Fulk,et al.  Saving the Phenomena: Requirements that Inductive Inference Machines Not Contradict Known Data , 1988, Inf. Comput..

[19]  Leonard Pitt,et al.  A polynomial-time algorithm for learning k-variable pattern languages from examples , 1989, COLT '89.

[20]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[21]  Klaus P. Jantke,et al.  Monotonic and Nonmonotonic Inductive Inference of Functions and Patterns , 1990, Nonmonotonic and Inductive Logic.

[22]  Wen-Guey Tzeng,et al.  Learning String Patterns and Tree Patterns from Examples , 1990, ML.

[23]  J. Feldman,et al.  Learning Automata from Ordered Examples , 1991 .

[24]  Thomas Zeugmann,et al.  Types of monotonic language learning and their characterization , 1992, COLT '92.