Learning in the presence of inaccurate information

Abstract The present paper considers the effects of introducing inaccuracies in a learner's environment in Gold's learning model of identification in the limit. Three kinds of inaccuracies are considered: presence of spurious data is modeled as learning from a noisy environment, missing data is modeled as learning from incomplete environment, and the presence of a mixture of both spurious and missing data is modeled as learning from imperfect environment. Two learning domains are considered, namely, identification of programs from graphs of computable functions and identification of grammars from positive data about recursively enumerable languages. Many hierarchies and tradeoffs resulting from the interplay between the number of errors allowed in the final hypotheses, the number of inaccuracies in the data, the types of inaccuracies, and the type of success criteria are derived. An interesting result is that in the context of function learning, incomplete data is strictly worse for learning than noisy data.

[1]  John Case,et al.  Machine Inductive Inference and Language Identification , 1982, ICALP.

[2]  Mark A. Fulk A study of inductive inference machines , 1986 .

[3]  Gisela Schäfer-Richter Some results in the theory of effective program synthesis: learning by defective information , 1985, Mathematical Methods of Specification and Synthesis of Software Systems.

[4]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[5]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[6]  Paul Young,et al.  An introduction to the general theory of algorithms , 1978 .

[7]  Kenneth Wexler,et al.  Formal Principles of Language Acquisition , 1980 .

[8]  Daniel N. Osherson,et al.  Ideal Learning Machines , 1982, Cogn. Sci..

[9]  Hartley Rogers,et al.  Gödel numberings of partial recursive functions , 1958, Journal of Symbolic Logic.

[10]  Jr. Hartley Rogers Theory of Recursive Functions and Effective Computability , 1969 .

[11]  John Case The power of vacillation , 1988, COLT '88.

[12]  Philip D. Laird,et al.  Learning from good data and bad , 1987 .

[13]  K. Wexler On extensional learnability , 1982, Cognition.

[14]  Rolf Wiehagen,et al.  Research in the theory of inductive inference by GDR mathematicians - A survey , 1980, Inf. Sci..

[15]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[16]  Gisela Schäfer Some results in the theory of effective program synthesis: learning by defective information , 1985 .

[17]  Mark A. Fulk,et al.  Learning in the presence of inaccurate information , 1989, COLT 1989.

[18]  Daniel N. Osherson,et al.  Criteria of Language Learning , 1982, Inf. Control..

[19]  Mark A. Fulk Prudence and Other Conditions on Formal Language Learning , 1990, Inf. Comput..

[20]  S. Pinker Formal models of language learning , 1979, Cognition.

[21]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[22]  D. Osherson,et al.  Learning theory and natural language , 1984, Cognition.

[23]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[24]  D. Osherson,et al.  A note on formal learning theory , 1982, Cognition.