The Complexity of Universal Text-Learners

The present work deals with language learning from text. It considers universal learners for classes of languages in models of additional information and analyzes their complexity in terms of Turing degrees. The following is shown: If the additional information is given by a set containing at least one index for each language from the class to be learned but no index for any language outside the class, then there is a universal learner having the same Turing degree as the inclusion problem for recursively enumerable sets. This result is optimal in the sense that any other successful learner has the same or higher Turing degree. If the additional information is given by the index set of the class of languages to be learned then there is a computable universal learner. Furthermore, if the additional information is presented as an upper bound on the size of some grammar that generates the language, then a high oracle is necessary and sufficient. Finally, it is shown that for the concepts of finite learning and learning from good examples, the index set of the class to be learned gives insufficient information due to the restrictive convergence constraints, these criteria need the jump of the index set instead of the index set itself. So, they have infinite access to the information of the index set in finite time.

[1]  Johan van Benthem,et al.  Handbook of Logic and Language , 1996 .

[2]  Jr. Hartley Rogers Theory of Recursive Functions and Effective Computability , 1969 .

[3]  P. Odifreddi Classical recursion theory , 1989 .

[4]  Sally A. Goldman,et al.  Teaching a Smarter Learner , 1996, J. Comput. Syst. Sci..

[5]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[6]  Shai Ben-David,et al.  Proceedings of the Third European Conference on Computational Learning Theory , 1997 .

[7]  H. Rice Classes of recursively enumerable sets and their decision problems , 1953 .

[8]  Rolf Wiehagen,et al.  Language Learning from Good Examples , 1994, AII/ALT.

[9]  William I. Gasarch,et al.  Learning via queries to an oracle , 1989, COLT '89.

[10]  Rae Baxter,et al.  Acknowledgments.-The authors would like to , 1982 .

[11]  Frank Stephan,et al.  Robust Learning with Infinite Additional Information , 1997, EuroCOLT.

[12]  Gianfranco Bilardi,et al.  On Uniform Learnability of Language Families , 1992, Inf. Process. Lett..

[13]  Robert I. Soare,et al.  Recursively enumerable sets and degrees - a study of computable functions and computability generated sets , 1987, Perspectives in mathematical logic.

[14]  Efim B. Kinber,et al.  On a Theory of Inductive Inference , 1977, FCT.

[15]  S. Shelah,et al.  Annals of Pure and Applied Logic , 1991 .

[16]  Manuel Blum,et al.  Inductive inference and unsolvability , 1991, Journal of Symbolic Logic.

[17]  Mark Steedman,et al.  In handbook of logic and language , 1997 .

[18]  Klaus P. Jantke,et al.  Natural Properties of Strategies Identifying Recursive Functions , 1979, J. Inf. Process. Cybern..

[19]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[20]  Arun Sharma,et al.  On the Non-Existence of Maximal Inference Degrees for Language Identification , 1993, Inf. Process. Lett..

[21]  Yasuhito Mukouchi,et al.  Characterization of Finite Identification , 1992, AII.

[22]  Rusins Freivalds,et al.  On the Power of Inductive Inference from Good Examples , 1993, Theor. Comput. Sci..

[23]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[24]  Daniel N. Osherson,et al.  A universal inductive inference machine , 1991, Journal of Symbolic Logic.

[25]  Daniel N. Osherson,et al.  Synthesizing Inductive Expertise , 1988, Inf. Comput..

[26]  Arun Sharma,et al.  Learning with the Knowledge of an Upper Bound on Program Size , 1993, Inf. Comput..

[27]  Scott Weinstein,et al.  Formal Learning Theory , 1997, Handbook of Logic and Language.

[28]  R. Soare Recursively enumerable sets and degrees , 1987 .

[29]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[30]  Stuart A. Kurtz,et al.  Extremes in the Degrees of Inferability , 1994, Ann. Pure Appl. Log..

[31]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[32]  Rolf Wiehagen,et al.  Inductive Inference with Additional Information , 1979, J. Inf. Process. Cybern..

[33]  John Case,et al.  Synthesizing enumeration techniques for language learning , 1996, COLT '96.