A general comparison of language learning from examples and from queries

In language learning, strong relationships between Gold-style models and query models have recently been observed: in some quite general setting Gold-style learners can be replaced by query learners and vice versa, without loss of learning capabilities. These 'equalities' hold in the context of learning indexable classes of recursive languages. Former studies on Gold-style learning of such indexable classes have shown that, in many settings, the enumerability of the target class and the recursiveness of its languages are crucial for learnability. Moreover, studying query learning, non-indexable classes have been mainly neglected up to now. So it is conceivable that the recently observed relations between Gold-style and query learning are not due to common structures in the learning processes in both models, but rather to the enumerability of the target classes or the recursiveness of their languages. In this paper, the analysis is lifted onto the context of learning arbitrary classes of recursively enumerable languages. Still, strong relationships between the approaches of Gold-style and query learning are proven, but there are significant changes to the former results. Though in many cases learners of one type can still be replaced by learners of the other type, in general this does not remain valid vice versa. All results hold even for learning classes of recursive languages, which indicates that the recursiveness of the languages is not crucial for the former 'equality' results. Thus we analyze how constraints on the algorithmic structure of the target class affect the relations between two approaches to language learning.

[1]  Dana Angluin,et al.  Queries revisited , 2001, Theoretical Computer Science.

[2]  Shirley Dex,et al.  JR 旅客販売総合システム(マルス)における運用及び管理について , 1991 .

[3]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[4]  Thomas Zeugmann,et al.  Language learning in dependence on the space of hypotheses , 1993, COLT '93.

[5]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[6]  Sandra Zilles,et al.  Comparison of Query Learning and Gold-Style Learning in Dependence of the Hypothesis Space , 2004, ALT.

[7]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[8]  M. W. Shields An Introduction to Automata Theory , 1988 .

[9]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[10]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[11]  Rajeev Motwani,et al.  Randomized Algorithms , 1995, SIGA.

[12]  Dick de Jongh,et al.  Angluin's theorem for indexed families of r.e. sets and applications , 1996, COLT '96.

[13]  John Case,et al.  Machine Inductive Inference and Language Identification , 1982, ICALP.

[14]  Jr. Hartley Rogers Theory of Recursive Functions and Effective Computability , 1969 .

[15]  Leslie G. Valiant,et al.  Computational limitations on learning from examples , 1988, JACM.

[16]  Thomas Zeugmann,et al.  A Guided Tour Across the Boundaries of Learning Recursive Languages , 1995, GOSLER Final Report.

[17]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[18]  Sanjay Jain,et al.  Gold-Style and Query Learning Under Various Constraints on the Target Class , 2005, ALT.

[19]  Manuel Blum,et al.  Inductive inference and unsolvability , 1991, Journal of Symbolic Logic.

[20]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[21]  Mark A. Fulk Prudence and Other Conditions on Formal Language Learning , 1990, Inf. Comput..

[22]  Sandra Zilles,et al.  Replacing Limit Learners with Equally Powerful One-Shot Query Learners , 2004, COLT.

[23]  Sandra Zilles,et al.  Relations between Gold-style learning and query learning , 2005, Inf. Comput..

[24]  Sanjay Jain,et al.  Learning Languages from Positive Data and Negative Counterexamples , 2004, ALT.

[25]  M. Kearns,et al.  Recent Results on Boolean Concept Learning , 1987 .