Vacillatory and BC learning on noisy data

Abstract The present work employs a model of noise introduced earlier by the third author. In this model noisy data nonetheless uniquely determines the true data: correct information occurs infinitely often while incorrect information occurs only finitely often. The present paper considers the effects of this form of noise on vacillatory and behaviorally correct learning of grammars – both from positive data alone and from informant (positive and negative data). For learning from informant, the noise, in effect, destroys negative data. Various noisy-data hierarchies are exhibited, which, in some cases, are known to collapse when there is no noise. Noisy behaviorally correct learning is shown to obey a very strong “subset principle”. It is shown, in many cases, how much power is needed to overcome the effects of noise. For example, the best we can do to simulate, in the presence of noise, the noise-free, no mind change cases takes infinitely many mind changes. One technical result is proved by a priority argument.

[1]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[2]  Efim B. Kinber,et al.  On a Theory of Inductive Inference , 1977, FCT.

[3]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[4]  John Case The power of vacillation , 1988, COLT '88.

[5]  Arun Sharma,et al.  Learning from Multiple Sources of Inaccurate Data , 1992, AII.

[6]  John Case,et al.  Learning Recursive Functions from Approximations , 1995, J. Comput. Syst. Sci..

[7]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[8]  Gianfranco Bilardi,et al.  Language Learning Without Overgeneralization , 1992, Theor. Comput. Sci..

[9]  Yasuhito Mukouchi,et al.  Characterization of Finite Identification , 1992, AII.

[10]  Frank Stephan Noisy Inference and Oracles , 1997, Theor. Comput. Sci..

[11]  Sanjay Jain,et al.  Open Problems in "Systems That Learn" , 1994, J. Comput. Syst. Sci..

[12]  R. Soare Recursively enumerable sets and degrees , 1987 .

[13]  John Case,et al.  Vacillatory Learning of Nearly Minimal Size Grammars , 1994, J. Comput. Syst. Sci..

[14]  John Case,et al.  Machine Learning of Higher-Order Programs , 1994, J. Symb. Log..

[15]  Arun Sharma,et al.  A Note on Batch and Incremental Learnability , 1998, J. Comput. Syst. Sci..

[16]  John Case,et al.  Machine Inductive Inference and Language Identification , 1982, ICALP.

[17]  William I. Gasarch,et al.  Learning via queries to an oracle , 1989, COLT '89.

[18]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[19]  Joseph H. Danks,et al.  Knowledge And Language , 1986 .

[20]  Sanjay Jain,et al.  Learning in the presence of inaccurate information , 1989, COLT '89.

[21]  L. Robertson,et al.  Normal sustained effects of selective attention are absent in schizophrenic patients withdrawn from medication , 1996, Psychiatry Research.

[22]  John Case,et al.  Synthesizing enumeration techniques for language learning , 1996, COLT '96.

[23]  Carl H. Smith,et al.  Inductive Inference: Theory and Methods , 1983, CSUR.

[24]  John Case,et al.  Complexity Issues for Vacillatory Function Identification , 1995, Inf. Comput..

[25]  K. Wexler The Subset Principle is an Intensional Principle , 1993 .

[26]  Thomas Zeugmann,et al.  Types of monotonic language learning and their characterization , 1992, COLT '92.

[27]  Thomas Zeugmann,et al.  Characterizations of Monotonic and Dual Monotonic Language Learning , 1995, Inf. Comput..

[28]  John Case,et al.  The Power of Vacillation in Language Learning , 1999, SIAM J. Comput..

[29]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[30]  Ya. M. Barzdin,et al.  Towards a Theory of Inductive Inference (in Russian) , 1973, MFCS.

[31]  P. Odifreddi Classical recursion theory , 1989 .

[32]  S. Kapur,et al.  Universal Grammar and Learnability Theory: The Case of Binding Domains and the ‘Subset Principle’ , 1993 .

[33]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[34]  Klaus P. Jantke,et al.  Analogical and Inductive Inference , 1986, Lecture Notes in Computer Science.

[35]  Robert C. Berwick,et al.  The acquisition of syntactic knowledge , 1985 .