Ordinal Mind Change Complexity of Language Identification

Abstract The approach of ordinal mind change complexity, introduced by Freivalds and Smith, uses (notations for) constructive ordinals to bound the number of mind changes made by a learning machine. This approach provides a measure of the extent to which a learning machine has to keep revising its estimate of the number of mind changes it will make before converging to a correct hypothesis for languages in the class being learned. Recently, this notion, which also yields a measure for the difficulty of learning a class of languages, has been used to analyze the learnability of rich concept classes. The present paper further investigates the utility of ordinal mind change complexity. It is shown that for identification from both positive and negative data and n ⩾ 1, the ordinal mind change complexity of the class of languages formed by unions of up to n + 1 pattern languages is only ω ×0 notn(n) (where notn(n) is a notation for n, ω is a notation for the least limit ordinal and ×0 represents ordinal multiplication). This result nicely extends an observation of Lange and Zeugmann that pattern languages can be identified from both positive and negative data with 0 mind changes. Existence of an ordinal mind change bound for a class of learnable languages can be seen as an indication of its learning “tractability”. Conditions are investigated under which a class has an ordinal mind change bound for identification from positive data. It is shown that an indexed family of languages has an ordinal mind change bound if it has finite elasticity and can be identified by a conservative machine. It is also shown that the requirement of conservative identification can be sacrificed for the purely topological requirement of M-finite thickness. Interaction between identification by monotonic strategies and existence of ordinal mind change bound is also investigated.

[1]  Paul Young,et al.  An introduction to the general theory of algorithms , 1978 .

[2]  Andris Ambainis.  The power of procrastination in inductive inference: How it depends on used ordinal notations , 1995, EuroCOLT.

[3]  Keith Wright.  Identification of unions of languages drawn from an identifiable class , 1989, COLT '89.

[4]  Thomas Zeugmann,et al.  Monotonic and Dual Monotonic Language Learning , 1996, Theor. Comput. Sci..

[5]  Shyam Kapur,et al.  Monotonic Language Learning , 1992, ALT.

[6]  G. Sacks.  Higher recursion theory , 1990 .

[7]  Arun Sharma,et al.  On the intrinsic complexity of language identification , 1994, COLT '94.

[8]  John Case,et al.  Not-So-Nearly-Minimal-Size Program Inference , 1995, GOSLER Final Report.

[9]  篠原 武.  Studies on inductive inference from positive data , 1986 .

[10]  Thomas Zeugmann,et al.  A Guided Tour Across the Boundaries of Learning Recursive Languages , 1995, GOSLER Final Report.

[11]  Dana Angluin,et al.  Finding Patterns Common to a Set of Strings , 1980, J. Comput. Syst. Sci..

[12]  Takeshi Shinohara,et al.  The correct definition of finite elasticity: corrigendum to identification of unions , 1991, COLT '91.

[13]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[14]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[15]  Carl H. Smith,et al.  On the Role of Procrastination in Machine Learning , 1993, Inf. Comput..

[16]  Heikki Mannila,et al.  MDL learning of unions of simple pattern languages from positive examples , 1995, EuroCOLT.

[17]  Robert E. Schapire,et al.  Pattern languages are not learnable , 1990, Annual Conference Computational Learning Theory.

[18]  Arun Sharma,et al.  Elementary formal systems, intrinsic complexity, and procrastination , 1997, COLT '96.

[19]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[20]  Arun Sharma,et al.  The Structure of Intrinsic Complexity of Learning , 1997, J. Symb. Log..

[21]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[22]  Akihiro Yamamoto,et al.  Algorithmic Learning Theory with Elementary Formal Systems , 1992 .

[23]  Stephen Cole Kleene,et al.  On notation for ordinal numbers , 1938, Journal of Symbolic Logic.

[24]  Takeshi Shinohara,et al.  Rich Classes Inferable from Positive Data: Length-Bounded Elementary Formal Systems , 1994, Inf. Comput..

[25]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[26]  Janis Barzdins,et al.  Towards a theory of inductive inference , 1973 .

[27]  Carl H. Smith,et al.  On the Complexity of Inductive Inference , 1986, Inf. Control..

[28]  Leonard Pitt,et al.  Inductive Inference, DFAs, and Computational Complexity , 1989, AII.

[29]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[30]  Raymond M. Smullyan.  Elementary formal systems , 1961 .

[31]  向内 康人,et al.  Inductive Inference of an Approximate Concept from Positive Data , 1993 .