Efficiency in the Identification in the Limit Learning Paradigm

The most widely used learning paradigm in Grammatical Inference was introduced in 1967 and is known as identification in the limit. An important issue that has been raised with respect to the original definition is the absence of efficiency bounds. Nearly fifty years after its introduction, it remains an open problem how to best incorporate a notion of efficiency and tractability into this framework. This chapter surveys the different refinements that have been developed and studied, and the challenges they face. Main results for each formalization, along with comparisons, are provided.

[1]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[2]  Alaa A. Kharbouch,et al.  Three models for the description of language , 1956, IRE Trans. Inf. Theory.

[3]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[4]  Dana Angluin,et al.  Finding Patterns Common to a Set of Strings , 1980, J. Comput. Syst. Sci..

[5]  Mary danced,et al.  Computational Theories of Learning and Developmental Psycholinguistics , 2011 .

[6]  Aryeh Kontorovich,et al.  On the learnability of shuffle ideals , 2012, J. Mach. Learn. Res..

[7]  T. Yokomori.  On Polynomial-Time Learnability in the Limit of Strictly Deterministic Automata , 1995, Machine Learning.

[8]  Tadao Kasami,et al.  On Multiple Context-Free Grammars , 1991, Theor. Comput. Sci..

[9]  Tim Oates,et al.  Learning k-Reversible Context-Free Grammars from Positive Structural Examples , 2002, ICML.

[10]  Sham M. Kakade,et al.  Identifiability and Unmixing of Latent Parse Trees , 2012, NIPS.

[11]  Colin de la Higuera.  Characteristic Sets for Polynomial Grammatical Inference , 1997 .

[12]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[13]  Dana Ron,et al.  On the learnability and usage of acyclic probabilistic finite automata , 1995, COLT '95.

[14]  Yasubumi Sakakibara,et al.  Efficient Learning of Context-Free Grammars from Positive Structural Examples , 1992, Inf. Comput..

[15]  S. Griffis.  EDITOR , 1997, Journal of Navigation.

[16]  Etsuji Tomita,et al.  A Fast Algorithm for Checking the Inclusion for Very Simple Deterministic Pushdown Automata , 1993 .

[17]  Alexander Clark,et al.  Learning trees from strings: a strong learning algorithm for some context-free grammars , 2013, J. Mach. Learn. Res..

[18]  Colin de la Higuera,et al.  Grammatical Inference: Learning Automata and Grammars , 2010 .

[19]  Yasuhiro Tajima,et al.  Polynomial time learning of simple deterministic languages via queries and a representative sample , 2004, Theor. Comput. Sci..

[20]  Ryo Yoshinaka,et al.  PAC Learning of Some Subclasses of Context-Free Grammars with Basic Distributional Properties from Positive Data , 2013, ALT.

[21]  Walter Daelemans.  Colin de la Higuera: Grammatical inference: learning automata and grammars , 2011, Machine Translation.

[22]  Alexander Clark,et al.  PAC-learnability of Probabilistic Deterministic Finite State Automata , 2004, J. Mach. Learn. Res..

[23]  Pedro García,et al.  IDENTIFYING REGULAR LANGUAGES IN POLYNOMIAL TIME , 1993 .

[24]  Friedrich Otto,et al.  String-Rewriting Systems , 1993, Text and Monographs in Computer Science.

[25]  Colin de la Higuera,et al.  LARS: A learning algorithm for rewriting systems , 2006, Machine Learning.

[26]  Federico Girosi,et al.  An Equivalence Between Sparse Approximation and Support Vector Machines , 1998, Neural Computation.

[27]  Enrique Vidal,et al.  What Is the Search Space of the Regular Inference? , 1994, ICGI.

[28]  Ryo Yoshinaka.  Learning efficiency of very simple grammars from positive data , 2009, Theor. Comput. Sci..

[29]  John Shawe-Taylor,et al.  A PAC analysis of a Bayesian estimator , 1997, COLT '97.

[30]  Ryo Yoshinaka,et al.  Distributional learning of parallel multiple context-free grammars , 2013, Machine Learning.

[31]  Thomas Zeugmann.  Can Learning in the Limit Be Done Efficiently? , 2003, Discovery Science.

[32]  Bernhard Steffen,et al.  Learning register automata: from languages to program structures , 2014, Machine Learning.

[33]  Alexander Clark,et al.  Polynomial Identification in the Limit of Substitutable Context-free Languages , 2005 .

[34]  John W. Carr,et al.  A Solution of the Syntactical Induction-Inference Problem for Regular Languages , 1978, Comput. Lang..

[35]  J. Langford.  Tutorial on Practical Prediction Theory for Classification , 2005, J. Mach. Learn. Res..

[36]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[37]  José M. Sempere,et al.  A Characterization of Even Linear Languages and its Application to the Learning Problem , 1994, ICGI.

[38]  Thomas Zeugmann,et al.  From learning in the limit to stochastic finite learning , 2006, Theor. Comput. Sci..

[39]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[40]  Andris Ambainis,et al.  Ordinal Mind Change Complexity of Language Identification , 1997, EuroCOLT.

[41]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[42]  Shalom Lappin,et al.  Linguistic Nativism and the Poverty of the Stimulus , 2011 .

[43]  Takashi Yokomori,et al.  Polynomial-time identification of very simple grammars from positive data , 2003, Theor. Comput. Sci..

[44]  Ryo Yoshinaka,et al.  Efficient learning of multiple context-free languages with multidimensional substitutability from positive data , 2011, Theor. Comput. Sci..

[45]  Hubert Comon,et al.  Tree automata techniques and applications , 1997 .

[46]  John Case,et al.  Difficulties in Forcing Fairness of Polynomial Time Inductive Inference , 2009, ALT.

[47]  Colin de la Higuera,et al.  Inferring Deterministic Linear Languages , 2002, COLT.

[48]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[49]  Yoshiyasu Ishigami,et al.  VC-dimensions of Finite Automata and Commutative Finite Automata with k Letters and n States , 1997, Discret. Appl. Math..

[50]  Ryo Yoshinaka,et al.  Identification in the Limit of k, l-Substitutable Context-Free Languages , 2008, ICGI.

[51]  Paul M. B. Vitányi,et al.  A Theory of Learning Simple Concepts Under Simple Distributions , 1989, COLT 1989.

[52]  Edward F. Moore,et al.  Gedanken-Experiments on Sequential Machines , 1956 .

[53]  Leonard Pitt,et al.  Inductive Inference, DFAs, and Computational Complexity , 1989, AII.

[54]  Leonor Becerra-Bonache,et al.  Learning DFA from Correction and Equivalence Queries , 2006, ICGI.