Robust Learning - Rich and Poor

A class C of recursive functions is called robustly learnable in the sense I (where I is any success criterion of learning) if not only C itself but even all transformed classes Θ(C) where Θ is any general recursive operator, are learnable in the sense I. It was already shown before, see [14,19], that for I = Ex (learning in the limit) robust learning is rich in that there are classes being both not contained in any recursively enumerable class of recursive functions and, nevertheless, robustly learnable. For several criteria I, the present paper makes much more precise where we can hope for robustly learnable classes and where we cannot. This is achieved in two ways. First, for I = Ex, it is shown that only consistently learnable classes can be uniformly robustly learnable. Second, some other learning types I are classified as to whether or not they contain rich robustly learnable classes. Moreover, the first results on separating robust learning from uniformly robust learning are derived.

[1]  Mark A. Fulk Robust separations in inductive inference , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[2]  Carl H. Smith,et al.  On the role of search for learning , 1989, COLT '89.

[3]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[4]  Christophe Costa Florêncio Consistent Identification in the Limit of Rigid Grammars from Strings Is NP-hard , 2002, ICGI.

[5]  Frank Stephan,et al.  Avoiding coding tricks by hyperrobust learning , 1999, Theor. Comput. Sci..

[6]  Steffen Lange,et al.  A Note on Polynominal-Time Inference of k-Variable Pattern Languages , 1990, Nonmonotonic and Inductive Logic.

[7]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory, Second Edition , 2000, Statistics for Engineering and Information Science.

[8]  Rusins Freivalds,et al.  Inductive Inference of Recursive Functions: Complexity Bounds , 1991, Baltic Computer Science.

[9]  Sanjay Jain,et al.  Robust Behaviorally Correct Learning , 1999, Inf. Comput..

[10]  Christophe Costa Florêncio Consistent identification in the limit of some of Penn and Buszkowski's classes is NP-hard , 1999, CLIN.

[11]  Rusins Freivalds,et al.  Inductive Inference of Recursive Functions: Qualitative Theory , 1991, Baltic Computer Science.

[12]  Rolf Wiehagen,et al.  Research in the theory of inductive inference by GDR mathematicians - A survey , 1980, Inf. Sci..

[13]  Eliana Minicozzi,et al.  Some Natural Properties of Strong-Identification in Inductive Inference , 1976, Theor. Comput. Sci..

[14]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[15]  Klaus P. Jantke,et al.  Combining Postulates of Naturalness in Inductive Inference , 1981, J. Inf. Process. Cybern..

[16]  Mark A. Fulk,et al.  Saving the Phenomena: Requirements that Inductive Inference Machines Not Contradict Known Data , 1988, Inf. Comput..

[17]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[18]  Werner Stein Consistent Polynominal Identification in the Limit , 1998, ALT.

[19]  Carl H. Smith,et al.  On the power of learning robustly , 1998, COLT' 98.

[20]  Jr. Hartley Rogers Theory of Recursive Functions and Effective Computability , 1969 .

[21]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[22]  Carl H. Smith,et al.  Robust Learning Is Rich , 2001, J. Comput. Syst. Sci..

[23]  Rolf Wiehagen,et al.  Learning and Consistency , 1995, GOSLER Final Report.

[24]  John Case,et al.  Robust learning aided by context , 1998, COLT' 98.

[25]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[26]  Rolf Wiehagen Limes-Erkennung rekursiver Funktionen durch spezielle Strategien , 1975, J. Inf. Process. Cybern..

[27]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[28]  Martin Anthony,et al.  Computational learning theory: an introduction , 1992 .

[29]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[30]  Rolf Wiehagen,et al.  Charakteristische Eigenschaften von erkennbaren Klassen rekursiver Funktionen , 1976, J. Inf. Process. Cybern..

[31]  Rolf Wiehagen,et al.  Ignoring data may be the only way to learn efficiently , 1994, J. Exp. Theor. Artif. Intell..

[32]  Rolf Wiehagen,et al.  Inductive Inference with Additional Information , 1979, J. Inf. Process. Cybern..

[33]  Carl H. Smith,et al.  Inductive Inference: Theory and Methods , 1983, CSUR.

[34]  Stuart A. Kurtz,et al.  A Refutation of Barzdins' Conjecture , 1989, AII.