We consider concept learning from examples. The learner receives - step by step - larger and larger initial segments of a sequence of examples describing an unknown target concept, processes these examples, and computes hypotheses. The learner is successful, if its hypotheses stabilize on a correct representation of the target concept. The underlying model is called identification in the limit. The present study concerns different versions of incremental learning in the limit. In contrast to the general case, now the learner has only limited access to the examples provided so far. In the special case of iterative learning, the learner builds its new hypotheses just on the basis of the current hypothesis and the next example, without having access to any of the other examples presented so far. In the case of bounded example-memory learning, the learner may in addition memorize up to an a priori fixed number of examples already presented. Formal studies have shown that restricting the accessibility of the input data results in a loss of learning power, i.e. there are concept classes learnable in the limit, but not identifiable by any incremental learner at all. The present analysis aims at illustrating this phenomenon and giving insights into the structure of concept classes incremental learners can cope with. Examples of identifiable and non-identifiable classes are given; different learning models are compared to one another with respect to the competence of the corresponding learners.
[1]
R. Rivest.
Learning Decision Lists
,
1987,
Machine Learning.
[2]
E. Mark Gold,et al.
Language Identification in the Limit
,
1967,
Inf. Control..
[3]
Leslie G. Valiant,et al.
A theory of the learnable
,
1984,
CACM.
[4]
Leslie G. Valiant,et al.
A theory of the learnable
,
1984,
STOC '84.
[5]
Dana Angluin,et al.
Finding Patterns Common to a Set of Strings
,
1980,
J. Comput. Syst. Sci..
[6]
G. G. Stokes.
"J."
,
1890,
The New Yale Book of Quotations.
[7]
Thomas Zeugmann,et al.
Incremental Learning from Positive Data
,
1996,
J. Comput. Syst. Sci..
[8]
Thomas Zeugmann,et al.
A Guided Tour Across the Boundaries of Learning Recursive Languages
,
1995,
GOSLER Final Report.
[9]
Rolf Wiehagen.
Limes-Erkennung rekursiver Funktionen durch spezielle Strategien
,
1975,
J. Inf. Process. Cybern..
[10]
Steffen Lange,et al.
On the power of incremental learning
,
2002,
Theor. Comput. Sci..
[11]
Dana Angluin,et al.
Inductive Inference of Formal Languages from Positive Data
,
1980,
Inf. Control..
[12]
Robert H. Sloan,et al.
BOOK REVIEW: "SYSTEMS THAT LEARN: AN INTRODUCTION TO LEARNING THEORY, SECOND EDITION", SANJAY JAIN, DANIEL OSHERSON, JAMES S. ROYER and ARUN SHARMA
,
2001
.
[13]
Jeffrey D. Ullman,et al.
Formal languages and their relation to automata
,
1969,
Addison-Wesley series in computer science and information processing.