Counting Extensional Differences in BC-Learning

Let BC be the model of behaviourally correct function learning as introduced by Barzdins [4] and Case and Smith [8]. We introduce a mind change hierarchy for BC, counting the number of extensional differences in the hypotheses of a learner. We compare the resulting models BC n to models from the literature and discuss confidence, team learning, and finitely defective hypotheses. Among other things, we prove that there is a tradeoff between the number of semantic mind changes and the number of anomalies in the hypotheses. We also discuss consequences for language learning. In particular we show that, in contrast to the case of function learning, the family of classes that are confidently BC-learnable from text is not closed under finite unions.

[1]  Carl H. Smith,et al.  The Power of Pluralism for Automatic Program Synthesis , 1982, JACM.

[2]  Frank Stephan,et al.  On the structure of degrees of inferability , 1993, COLT '93.

[3]  Daniel N. Osherson,et al.  Aggregating Inductive Expertise , 1986, Inf. Control..

[4]  Carl H. Smith,et al.  On the Role of Procrastination in Machine Learning , 1993, Inf. Comput..

[5]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[6]  Piergiorgio Odifreddi.  Inductive inference of total functions , 1996 .

[7]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[8]  Arun Sharma,et al.  Generalized notions of mind change complexity , 1997, COLT '97.

[9]  John Case,et al.  Complexity Issues for Vacillatory Function Identification , 1995, Inf. Comput..

[10]  Sebastiaan Terwijn.  Extensional set learning (extended abstract) , 1999, COLT '99.

[11]  Robert H. Sloan,et al.  BOOK REVIEW: "SYSTEMS THAT LEARN: AN INTRODUCTION TO LEARNING THEORY, SECOND EDITION", SANJAY JAIN, DANIEL OSHERSON, JAMES S. ROYER and ARUN SHARMA , 2001 .

[12]  Carl H. Smith,et al.  Inductive Inference with Procrastination: Back to Definitions , 1999, Fundam. Informaticae.

[13]  Andris Ambainis,et al.  Ordinal Mind Change Complexity of Language Identification , 1997, Theor. Comput. Sci..

[14]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[15]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[16]  Sebastiaan A. Terwijn,et al.  Extensional Set Learning , 2000 .

[17]  R. Soare.  Recursively enumerable sets and degrees , 1987 .

[18]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[19]  P. Odifreddi.  Classical recursion theory , 1989 .

[20]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .