Iterative learning of simple external contextual languages

It is investigated for which choice of a parameter q, denoting the number of contexts, the class of simple external contextual languages is iteratively learnable. On one hand, the class admits, for all values of q, polynomial time learnability provided an adequate choice of the hypothesis space is given. On the other hand, additional constraints like consistency and conservativeness or the use of a one-one hypothesis space changes the picture -- iterative learning limits the long term memory of the learner to the current hypothesis and these constraints further hinder storage of information via padding of this hypothesis. It is shown that if q> 3, then simple external contextual languages are not iteratively learnable using a class preserving one-one hypothesis space, while for q= 1 it is iteratively learnable, even in polynomial time. For the intermediate levels, there is some indication that iterative learnability using a class preserving one-one hypothesis space might depend on the size of the alphabet. It is also investigated for which choice of the parameters, the simple external contextual languages can be learnt by a consistent and conservative iterative learner.

[1]  Alaa A. Kharbouch,et al.  Three models for the description of language , 1956, IRE Trans. Inf. Theory.

[2]  Mark Steedman,et al.  Dependency and Coordination in the Grammar of Dutch and English , 1985 .

[3]  A. Mateescu,et al.  Contexts and the Concept of Mild Context-Sensitivity , 2003 .

[4]  Leonor Becerra-Bonache,et al.  Inferring Grammars for Mildly Context Sensitive Languages in Polynomial-Time , 2006, ICGI.

[5]  P. Odifreddi.  Classical recursion theory , 1989 .

[6]  Alexis Manaster Ramer.  Some uses and abuses of mathematics in linguistics , 1999 .

[7]  Carl H. Smith,et al.  On the Intrinsic Complexity of Learning , 1995, Inf. Comput..

[8]  Aravind K. Joshi,et al.  Natural language parsing: Tree adjoining grammars: How much context-sensitivity is required to provide reasonable structural descriptions? , 1985 .

[9]  Leonor Becerra-Bonache,et al.  Learning Mild Context-Sensitiveness: Toward Understanding Children's Language Learning , 2004, ICGI.

[10]  Thomas Zeugmann,et al.  Language learning in dependence on the space of hypotheses , 1993, COLT '93.

[11]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[12]  Kelly Roach,et al.  Formal Properties of Head Grammars , 1987 .

[13]  Takeshi Shinohara,et al.  Rich Classes Inferable from Positive Data: Length-Bounded Elementary Formal Systems , 1994, Inf. Comput..

[14]  Rolf Wiehagen.  Limes-Erkennung rekursiver Funktionen durch spezielle Strategien , 1975, J. Inf. Process. Cybern..

[15]  C. Culy.  The complexity of the vocabulary of Bambara , 1985 .

[16]  Sanjay Jain.  Incremental Concept Learning for Bounded Data Mining Incremental Concept Learning for Bounded Data Mining , 1997 .

[17]  Ryo Yoshinaka.  Learning efficiency of very simple grammars from positive data , 2009, Theor. Comput. Sci..

[18]  Carlos Martín-Vide,et al.  Contextual Grammars as Generative Models of Natural Languages , 1998, Comput. Linguistics.

[19]  Rohit Parikh,et al.  On Context-Free Languages , 1966, JACM.

[20]  Leonard Pitt,et al.  Inductive Inference, DFAs, and Computational Complexity , 1989, AII.

[21]  Rolf Wiehagen,et al.  Polynomial-time inference of arbitrary pattern languages , 2009, New Generation Computing.

[22]  Thomas Zeugmann,et al.  Incremental Learning from Positive Data , 1996, J. Comput. Syst. Sci..

[23]  Carl H. Smith,et al.  On the role of procrastination for machine learning , 1992, COLT '92.

[24]  Solomon Marcus,et al.  Contextual Grammars , 1969, COLING.

[25]  Aravind K. Joshi,et al.  Tree-Adjoining Grammars , 1997, Handbook of Formal Languages.

[26]  Kenneth Wexler,et al.  Formal Principles of Language Acquisition , 1980 .

[27]  Carl H. Smith,et al.  On the Intrinsic Complexity of Learning , 1995, Inf. Comput..

[28]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[29]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[30]  John Case,et al.  Parallelism Increases Iterative Learning Power , 2007, ALT.

[31]  John Case,et al.  Machine Inductive Inference and Language Identification , 1982, ICALP.

[32]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[33]  Henning Fernau,et al.  External Contextual and Conditional Languages , 2000, Recent Topics in Mathematical and Computational Linguistics.

[34]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[35]  Stuart M. Shieber,et al.  Evidence against the context-freeness of natural language , 1985 .

[36]  Daniel N. Osherson,et al.  Criteria of Language Learning , 1982, Inf. Control..

[37]  Mark A. Fulk.  Prudence and Other Conditions on Formal Language Learning , 1990, Inf. Comput..

[38]  Stanley Peters,et al.  Cross-Serial Dependencies in Dutch , 1982 .

[39]  John Case,et al.  Results on memory-limited U-shaped learning , 2007, Inf. Comput..

[40]  Leonor Becerra Bonache.  On the learnibility of Mildly Context-Sensitive languages using positive data and correction queries , 2006 .

[41]  Gheorghe Paun,et al.  Marcus Contextual Grammars , 1994, Bull. EATCS.