Variants of iterative learning

We investigate the principal learning capabilities of iterative learners in some more details. Thereby, we confine ourselves to study the learnability of indexable concept classes. The general scenario of iterative learning is as follows. An iterative learner successively takes as input one element of a text (an informant) for a target concept as well as its previously made hypothesis and outputs a new hypothesis about the target concept. The sequence of hypotheses has to converge to a hypothesis correctly describing the target concept.We study two variants of this basic scenario and compare the learning capabilities of all resulting models of iterative learning to one another as well to the standard learning models finite inference, conservative identification, and learning in the limit.First, we consider the case that an iterative learner has to learn from fat texts (fat informants), only. In this setting, it is guaranteed that relevant information is, in principle, accessible at any time in the learning process. Second, we study a variant of iterative learning, where an iterative learner is supposed to learn no matter which initial hypothesis is actually chosen. This variant is suited to describe scenarios that are typical for case-based reasoning.

[1]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[2]  Rokia Missaoui,et al.  An Incremental Concept Formation Approach for Learning from Databases , 1994, Theor. Comput. Sci..

[3]  Thomas Zeugmann,et al.  Incremental Learning from Positive Data , 1996, J. Comput. Syst. Sci..

[4]  Sanjay Jain,et al.  Open Problems in "Systems That Learn" , 1994, J. Comput. Syst. Sci..

[5]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[6]  Robert H. Sloan,et al.  BOOK REVIEW: "SYSTEMS THAT LEARN: AN INTRODUCTION TO LEARNING THEORY, SECOND EDITION", SANJAY JAIN, DANIEL OSHERSON, JAMES S. ROYER and ARUN SHARMA , 2001 .

[7]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[8]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[9]  J. Feldman,et al.  Learning Automata from Ordered Examples , 1991 .

[10]  Thomas Zeugmann,et al.  Language learning in dependence on the space of hypotheses , 1993, COLT '93.

[11]  Thomas Zeugmann,et al.  Set-Driven and Rearrangement-Independent Learning of Recursive Languages , 1994, AII/ALT.

[12]  Rolf Wiehagen.  Limes-Erkennung rekursiver Funktionen durch spezielle Strategien , 1975, J. Inf. Process. Cybern..

[13]  Paul E. Utgoff,et al.  Incremental Induction of Decision Trees , 1989, Machine Learning.

[14]  Kenneth Wexler,et al.  Formal Principles of Language Acquisition , 1980 .

[15]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[16]  Thomas Zeugmann,et al.  Types of monotonic language learning and their characterization , 1992, COLT '92.

[17]  Ryszard S. Michalski,et al.  Selecting Examples for Partial Memory Learning , 2000, Machine Learning.

[18]  Pat Langley,et al.  Models of Incremental Concept Formation , 1990, Artif. Intell..

[19]  Sanjay Jain.  Incremental Concept Learning for Bounded Data Mining Incremental Concept Learning for Bounded Data Mining , 1997 .

[20]  Dana Angluin,et al.  Finding Patterns Common to a Set of Strings , 1980, J. Comput. Syst. Sci..

[21]  Klaus P. Jantke,et al.  Combining Postulates of Naturalness in Inductive Inference , 1981, J. Inf. Process. Cybern..

[22]  Thomas Zeugmann,et al.  A Guided Tour Across the Boundaries of Learning Recursive Languages , 1995, GOSLER Final Report.

[23]  Yuri L. Ershov,et al.  Theory of Numberings , 1999, Handbook of Computability Theory.

[24]  Frank Stephan,et al.  Language learning from texts (extended abstract): mind changes, limited memory and monotonicity , 1995, COLT '95.

[25]  Steffen Lange,et al.  On the Strength of Incremental Learning , 1999, ALT.

[26]  Frank Stephan,et al.  Noisy Inference and Oracles , 1995, Theor. Comput. Sci..

[27]  Janet L. Kolodner,et al.  An introduction to case-based reasoning , 1992, Artificial Intelligence Review.

[28]  R. Rivest.  Learning Decision Lists , 1987, Machine Learning.

[29]  Frank Stephan,et al.  Language Learning from Texts: Mind Changes, Limited Memory and Monotonicity (Extended Abstract). , 1995, COLT 1995.

[30]  Jeffrey D. Ullman,et al.  Formal languages and their relation to automata , 1969, Addison-Wesley series in computer science and information processing.