Too Much Can be Too Much for Learning Efficiently

In designing learning algorithms it seems quite reasonable to construct them in a way such that all data the algorithm already has obtained are correctly and completely reflected in the description the algorithm outputs on these data. However, this approach may totally fail, i.e., it may lead to the unsolvability of the learning problem, or it may exclude any efficient solution of it. In particular, we present a natural learning problem and prove that it can be solved in polynomial time if and only if the algorithm is allowed to ignore data.

[1]  Takeshi Shinohara,et al.  Polynomial Time Inference of Extended Regular Pattern Languages , 1983, RIMS Symposium on Software Science and Engineering.

[2]  Carl H. Smith,et al.  Inductive Inference: Theory and Methods , 1983, CSUR.

[3]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[4]  Rolf Wiehagen,et al.  Characterization Problems in the Theory of Inductive Inference , 1978, ICALP.

[5]  Thomas Zeugmann,et al.  A-posteriori Characterizations in Inductive Inference of Recursive Functions , 1983, J. Inf. Process. Cybern..

[6]  Klaus P. Jantke,et al.  Combining Postulates of Naturalness in Inductive Inference , 1981, J. Inf. Process. Cybern..

[7]  Mark A. Fulk,et al.  Saving the Phenomena: Requirements that Inductive Inference Machines Not Contradict Known Data , 1988, Inf. Comput..

[8]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[9]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[10]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[11]  Wen-Guey Tzeng,et al.  Learning String Patterns and Tree Patterns from Examples , 1990, ML.

[12]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[13]  Thomas Zeugmann,et al.  Monotonic Versus Nonmonotonic Language Learning , 1991, Nonmonotonic and Inductive Logic.

[14]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[15]  Leonard Pitt,et al.  A polynomial-time algorithm for learning k-variable pattern languages from examples , 1989, COLT '89.

[16]  Thomas Zeugmann,et al.  Types of monotonic language learning and their characterization , 1992, COLT '92.

[17]  J. Feldman,et al.  Learning Automata from Ordered Examples , 1991 .

[18]  Dana Angluin,et al.  Finding Patterns Common to a Set of Strings , 1980, J. Comput. Syst. Sci..

[19]  Rolf Wiehagen.  Limes-Erkennung rekursiver Funktionen durch spezielle Strategien , 1975, J. Inf. Process. Cybern..

[20]  Jerome A. Feldman,et al.  Learning Automata from Ordered Examples , 1991, Mach. Learn..

[21]  Robert Nix,et al.  Editing by example , 1985, POPL '84.

[22]  Stuart C. Shapiro,et al.  Encyclopedia of artificial intelligence, vols. 1 and 2 (2nd ed.) , 1992 .