Robust learning--rich and poor

A class b of recursive functions is called robustly learnable in the sense I (where I is any success criterion of learning) if not only b itself but even all transformed classes Θ(b), where Θ is any general recursive operator, are learnable in the sense I. It was already shown before, see Fulk (in: 31st Annual IEEE Symposium on Foundation of Computer Science, IEEE Computer Soc. Press, Silver Spring, MD 1990, pp. 405-410), Jain et al. (J. Comput. System Sci. 62 (2001) 178), that for I=Ex (learning in the limit) robust learning is rich in that there are classes being both not contained in any recursively enumerable class of recursive functions and, nevertheless, robustly learnable. For several criteria I, the present paper makes much more precise where we can hope for robustly learnable classes and where we cannot. This is achieved in two ways. First, for I=Ex, it is shown that only consistently learnable classes can be uniformly robustly learnable. Second, some other learning types I are classified as to whether or not they contain rich robustly learnable classes. Moreover, the first results on separating robust learning from uniformly robust learning are derived.

[1]  Christophe Costa Florêncio Consistent identification in the limit of some of Penn and Buszkowski's classes is NP-hard , 1999, CLIN.

[2]  Eliana Minicozzi,et al.  Some Natural Properties of Strong-Identification in Inductive Inference , 1976, Theor. Comput. Sci..

[3]  Steffen Lange,et al.  A Note on Polynominal-Time Inference of k-Variable Pattern Languages , 1990, Nonmonotonic and Inductive Logic.

[4]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[5]  John Case,et al.  Periodicity in generations of automata , 1974, Mathematical systems theory.

[6]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[7]  Rolf Wiehagen Limes-Erkennung rekursiver Funktionen durch spezielle Strategien , 1975, J. Inf. Process. Cybern..

[8]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[9]  Thomas Zeugmann,et al.  On Barzdin's Conjecture , 1986, AII.

[10]  Werner Stein Consistent Polynominal Identification in the Limit , 1998, ALT.

[11]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[12]  Martin Anthony,et al.  Computational learning theory: an introduction , 1992 .

[13]  Carl H. Smith,et al.  Robust Learning Is Rich , 2001, J. Comput. Syst. Sci..

[14]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[15]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[16]  Robert H. Sloan,et al.  BOOK REVIEW: "SYSTEMS THAT LEARN: AN INTRODUCTION TO LEARNING THEORY, SECOND EDITION", SANJAY JAIN, DANIEL OSHERSON, JAMES S. ROYER and ARUN SHARMA , 2001 .

[17]  Klaus P. Jantke,et al.  Combining Postulates of Naturalness in Inductive Inference , 1981, J. Inf. Process. Cybern..

[18]  Mark A. Fulk,et al.  Saving the Phenomena: Requirements that Inductive Inference Machines Not Contradict Known Data , 1988, Inf. Comput..

[19]  Rolf Wiehagen,et al.  Charakteristische Eigenschaften von erkennbaren Klassen rekursiver Funktionen , 1976, J. Inf. Process. Cybern..

[20]  Sanjay Jain,et al.  Robust Behaviorally Correct Learning , 1999, Inf. Comput..

[21]  Rolf Wiehagen,et al.  Ignoring data may be the only way to learn efficiently , 1994, J. Exp. Theor. Artif. Intell..

[22]  Rolf Wiehagen,et al.  Inductive Inference with Additional Information , 1979, J. Inf. Process. Cybern..

[23]  Carl H. Smith,et al.  Inductive Inference: Theory and Methods , 1983, CSUR.

[24]  Stuart A. Kurtz,et al.  A Refutation of Barzdins' Conjecture , 1989, AII.

[25]  Carl H. Smith,et al.  On the role of search for learning , 1989, COLT '89.

[26]  Rusins Freivalds,et al.  Inductive Inference of Recursive Functions: Complexity Bounds , 1991, Baltic Computer Science.

[27]  Mark A. Fulk Robust separations in inductive inference , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[28]  Jr. Hartley Rogers Theory of Recursive Functions and Effective Computability , 1969 .

[29]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[30]  Rolf Wiehagen,et al.  Learning and Consistency , 1995, GOSLER Final Report.

[31]  Rusins Freivalds,et al.  Inductive Inference of Recursive Functions: Qualitative Theory , 1991, Baltic Computer Science.

[32]  Rolf Wiehagen,et al.  Research in the theory of inductive inference by GDR mathematicians - A survey , 1980, Inf. Sci..

[33]  Frank Stephan,et al.  Avoiding coding tricks by hyperrobust learning , 1999, Theor. Comput. Sci..