Learning and classifying

Abstract We define and study a learning paradigm that sits between identification in the limit and classification. More precisely, we expect a learner to determine in the limit which members of a finite set D of possible data belong to a target language L , where D is arbitrary. So as D becomes larger and larger, the task becomes closer and closer to identifying L . But as D is always finite and L can be infinite, it can still be expected that Ex- and BC-learning are often more difficult than performing this classification task. The paper supports this intuition and makes it precise, taking into account desirable constraints on how the learner behaves, such as bounding the number of mind changes and being conservative. Special attention is given to various forms of consistency. In particular, we might not only require consistency between the members of D to classify, the current data σ and a language L , but also consistency between larger sets of possible data to classify (supersets of D ) and the same σ and L : whereas in the classical paradigms of inductive inference or classification, only the available data can grow, here both the available data and the set of possible data to classify can grow. We provide a fairly comprehensive set of results, many of which are optimal, that demonstrate the fruitfulness of the approach and the richness of the paradigm.

[1]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[2]  John Case,et al.  Periodicity in generations of automata , 1974, Mathematical systems theory.

[3]  Rusins Freivalds,et al.  An inductive inference approach to classification , 1994, J. Exp. Theor. Artif. Intell..

[4]  Jr. Hartley Rogers Theory of Recursive Functions and Effective Computability , 1969 .

[5]  Frank Stephan,et al.  Classification using information , 1994, Annals of Mathematics and Artificial Intelligence.

[6]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[7]  Klaus P. Jantke,et al.  Combining Postulates of Naturalness in Inductive Inference , 1981, J. Inf. Process. Cybern..

[8]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[9]  Daniel N. Osherson,et al.  Criteria of Language Learning , 1982, Inf. Control..

[10]  John Case,et al.  On the classification of recursive languages , 2004, Inf. Comput..

[11]  W. Gasarch,et al.  Bounded Queries in Recursion Theory , 1998 .

[12]  Carl H. Smith,et al.  Generalization versus classification , 1995, J. Exp. Theor. Artif. Intell..

[13]  Thomas Zeugmann,et al.  Learning recursive functions: A survey , 2008, Theor. Comput. Sci..

[14]  Sanjay Jain,et al.  Absolute versus probabilistic classification in a logical setting , 2005, Theoretical Computer Science.

[15]  Sanjay Jain On an open problem in classification of languages , 2001, J. Exp. Theor. Artif. Intell..

[16]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[17]  John Case,et al.  Machine Inductive Inference and Language Identification , 1982, ICALP.

[18]  Carl H. Smith,et al.  Classifying Predicates and Languages , 1997, Int. J. Found. Comput. Sci..

[19]  Rolf Wiehagen,et al.  Charakteristische Eigenschaften von erkennbaren Klassen rekursiver Funktionen , 1976, J. Inf. Process. Cybern..

[20]  Frank Stephan,et al.  On one-sided versus two-sided classification , 2001, Arch. Math. Log..

[21]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..