Topics in Grammatical Inference

This book explains advanced theoretical and application-related issues in grammatical inference, a research area inside the inductive inference paradigm for machine learning. The first three chapters of the book deal with issues regarding theoretical learning frameworks; the next four chapters focus on the main classes of formal languages according to Chomsky's hierarchy, in particular regular and context-free languages; and the final chapter addresses the processing of biosequences. The topics chosen are of foundational interest with relatively mature and established results, algorithms and conclusions. The book will be of value to researchers and graduate students in areas such as theoretical computer science, machine learning, computational linguistics, bioinformatics, and cognitive psychology who are engaged with the study of learning, especially of the structure underlying the concept to be learned. Some knowledge of mathematics and theoretical computer science, including formal language theory, automata theory, formal grammars, and algorithmics, is a prerequisite for reading this book.

[1]  Yusuke Suzuki,et al.  Polynomial Time Inductive Inference of Ordered Tree Languages with Height-Constrained Variables from Positive Data , 2004, PRICAI.

[2]  Qun Liu,et al.  Forest-Based Translation , 2008, ACL.

[3]  Kaizhong Zhang,et al.  Comparing multiple RNA secondary structures using tree comparisons , 1990, Comput. Appl. Biosci..

[4]  Cristina Tîrnauca,et al.  Learning Regular Tree Languages from Correction and Equivalence Queries , 2007, J. Autom. Lang. Comb..

[5]  Thomas Schwentick,et al.  Automata for XML - A survey , 2007, J. Comput. Syst. Sci..

[6]  Satoru Miyano.  Learning Theory Toward Genome Informatics , 1995, IEICE Trans. Inf. Syst..

[7]  Efim B. Kinber,et al.  On Learning Regular Expressions and Patterns Via Membership and Correction Queries , 2008, ICGI.

[8]  Ryo Yoshinaka,et al.  Learning Mildly Context-Sensitive Languages with Multidimensional Substitutability from Positive Data , 2009, ALT.

[9]  Libor Polák.  Minimalizations of NFA Using the Universal Automaton , 2004, CIAA.

[10]  Daniel Fredouille,et al.  Unambiguous Automata Inference by Means of State-Merging Methods , 2003, ECML.

[11]  Ryo Yoshinaka,et al.  PAC Learning of Some Subclasses of Context-Free Grammars with Basic Distributional Properties from Positive Data , 2013, ALT.

[12]  Joachim Niehren,et al.  A learning algorithm for top-down XML transformations , 2010, PODS '10.

[13]  Henning Fernau,et al.  Learning tree languages from text , 2002, RAIRO Theor. Informatics Appl..

[14]  Miles Osborne,et al.  Statistical Machine Translation , 2010, Encyclopedia of Machine Learning and Data Mining.

[15]  Stefan C. Kremer,et al.  Inducing Grammars from Sparse Data Sets: A Survey of Algorithms and Results , 2003, J. Mach. Learn. Res..

[16]  Henning Fernau.  Identification of Function Distinguishable Languages , 2000, ALT.

[17]  K. G. Subramanian,et al.  On the Inference of Linear Single Tree Grammars from Positive Structural Information , 2007, IICAI.

[18]  David J. Weir,et al.  The equivalence of four extensions of context-free grammars , 1994, Mathematical systems theory.

[19]  Keith Wright.  Identification of unions of languages drawn from an identifiable class , 1989, COLT '89.

[20]  Takayoshi Shoudai,et al.  Polynomial Time Inductive Inference of Regular Term Tree Languages from Positive Data , 1997, ALT.

[21]  Yusuke Suzuki,et al.  Discovery of Maximally Frequent Tag Tree Patterns with Height-Constrained Variables from Semistructured Web Documents , 2005, International Workshop on Challenges in Web Information Retrieval and Integration.

[22]  Aurélien Lemay,et al.  Learning Regular Languages Using RFSA , 2001, ALT.

[23]  Kaizhong Zhang,et al.  An Algorithm for Finding the Largest Approximately Common Substructures of Two Trees , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Dan Klein,et al.  Learning Accurate, Compact, and Interpretable Tree Annotation , 2006, ACL.

[25]  Van Helden,et al.  Case and Gender. Concept Formation between Morphology and Syntax , 1993 .

[26]  Philip N. Klein,et al.  Indexing based on edit-distance matching of shape graphs , 1998, Other Conferences.

[27]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[28]  Jacques Sakarovitch,et al.  The universal automaton , 2008, Logic and Automata.

[29]  Damián López,et al.  Syntactic Pattern Recognition by Error Correcting Analysis on Tree Automata , 2000, SSPR/SPR.

[30]  Takeshi Shinohara,et al.  Rich Classes Inferable from Positive Data: Length-Bounded Elementary Formal Systems , 1994, Inf. Comput..

[31]  Jean-Marc Champarnaud,et al.  NFA reduction algorithms by means of regular inequalities , 2004, Theor. Comput. Sci..

[32]  Takashi Yokomori,et al.  Learning non-deterministic finite automata from queries and counterexamples , 1994, Machine Intelligence 13.

[33]  Ashish Vaswani,et al.  Rule Markov Models for Fast Tree-to-String Translation , 2011, ACL.

[34]  Joachim Niehren,et al.  Logics and Automata for Totally Ordered Trees , 2008, RTA.

[35]  Damián López,et al.  Error-correcting tree language inference , 2002, Pattern Recognit. Lett..

[36]  Thomas Schwentick,et al.  Automata-and Logic-Based Pattern Languages for Tree-Structured Data , 2001, Semantics in Databases.

[37]  Simon M. Lucas,et al.  A Comparison of Syntactic and Statistical Techniques for Off-Line OCR , 1994, ICGI.

[38]  Joachim Niehren,et al.  Query induction with schema-guided pruning strategies , 2013, J. Mach. Learn. Res..

[39]  Gordon Plotkin,et al.  A Note on Inductive Generalization , 2008 .

[40]  James W. Thatcher,et al.  Characterizing Derivation Trees of Context-Free Grammars through a Generalization of Finite Automata Theory , 1967, J. Comput. Syst. Sci..

[41]  Erkki Mäkinen.  On inferring linear single-tree languages , 2000, Inf. Process. Lett..

[42]  Ryo Yoshinaka,et al.  Polynomial-Time Identification of Multiple Context-Free Languages from Positive Data and Membership Queries , 2010, ICGI.

[43]  Stephen Kwek,et al.  On learning unions of pattern languages and tree patterns in the mistake bound model , 2002, Theor. Comput. Sci..

[44]  Takashi Yokomori,et al.  Polynomial-time MAT Learning of C-Deterministic Context-free Grammars , 1993 .

[45]  Ariadna Quattoni,et al.  Spectral Learning for Non-Deterministic Dependency Parsing , 2012, EACL.

[46]  Anna Kasprzik,et al.  Four one-shot learners for regular tree languages and their polynomial characterizability , 2013, Theor. Comput. Sci..

[47]  Rudolf Freund,et al.  Bounded Parallelism in Array Grammars Used for Character Recognition , 1996, SSPR.

[48]  Francisco Casacuberta,et al.  Active learning for interactive machine translation , 2012, EACL.

[49]  José Ruiz Ochando.  Familias de lenguajes explorables: inferencia inductiva y caracterización algebraica , 1998 .

[50]  Enrique Vidal,et al.  Learning Subsequential Transducers for Pattern Recognition Interpretation Tasks , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Terry Caelli,et al.  Inexact Multisubgraph Matching Using Graph Eigenspace and Clustering Models , 2002, SSPR/SPR.

[52]  Olgierd Unold,et al.  How to use crowding selection in grammar-based classifier system , 2005, 5th International Conference on Intelligent Systems Design and Applications (ISDA'05).

[53]  M. Kearns,et al.  On the complexity of teaching , 1991, COLT '91.

[54]  Aravind K. Joshi,et al.  Tree Adjunct Grammars , 1975, J. Comput. Syst. Sci..

[55]  Timo Knuutila,et al.  The Inference of Tree Languages from Finite Samples: An Algebraic Approach , 1994, Theor. Comput. Sci..

[56]  Kevin J. Lang.  Random DFA's can be approximately learned from sparse uniform examples , 1992, COLT '92.

[57]  Hiroshi Matsui,et al.  Pair stochastic tree adjoining grammars for aligning and predicting pseudoknot RNA structures , 2004, Proceedings. 2004 IEEE Computational Systems Bioinformatics Conference, 2004. CSB 2004..

[58]  Benjamin B. Kimia,et al.  Shapes, shocks, and deformations I: The components of two-dimensional shape and the reaction-diffusion space , 1995, International Journal of Computer Vision.

[59]  Frank Neven,et al.  Automata, Logic, and XML , 2002, CSL.

[60]  DANA ANGLUIN,et al.  On the Complexity of Minimum Inference of Regular Sets , 1978, Inf. Control..

[61]  Enrique Vidal,et al.  Inference of k-Testable Languages in the Strict Sense and Application to Syntactic Pattern Recognition , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[62]  Damián López,et al.  Inference of K-Piecewise Testable Tree Languages , 2003 .

[63]  Pedro García,et al.  A Comparative Study of Two Algorithms for Automata Identification , 2000, ICGI.

[64]  Ryo Yoshinaka,et al.  Identification in the Limit of k, l-Substitutable Context-Free Languages , 2008, ICGI.

[65]  Kevin Knight,et al.  An Overview of Probabilistic Tree Transducers for Natural Language Processing , 2005, CICLing.

[66]  Gloria Inés,et al.  Estudio de la mezcla de estados determinista y no determinista en el diseño de algoritmos para inferencia gramatical de lenguajes regulares. , 2011 .

[67]  Ali Shokoufandeh,et al.  Shock Graphs and Shape Matching , 1998, International Journal of Computer Vision.

[68]  José Oncina,et al.  Learning Stochastic Regular Grammars by Means of a State Merging Method , 1994, ICGI.

[69]  Christof Löding.  Basics on Tree Automata , 2012, Modern Applications of Automata Theory.

[70]  Henning Fernau,et al.  Algorithms for learning regular expressions from positive data , 2009, Inf. Comput..

[71]  Andreas Maletti.  Learning Deterministically Recognizable Tree Series - Revisited , 2007, CAI.

[72]  William A. Gale,et al.  A sequential algorithm for training text classifiers , 1994, SIGIR '94.

[73]  Chris Quirk,et al.  Dependency Treelet Translation: Syntactically Informed Phrasal SMT , 2005, ACL.

[74]  J. Oncina.  Inference of recognizable tree sets , 2003 .

[75]  Edward P. Stabler,et al.  Derivational Minimalism , 1996, LACL.

[76]  Enrique Vidal,et al.  Identification of DFA: data-dependent vs data-independent algorithms , 1996, ICGI.

[77]  E. Mark Gold,et al.  Complexity of Automaton Identification from Given Data , 1978, Inf. Control..

[78]  Pedro García,et al.  Learning in varieties of the form V*LI from positive data , 2006, Theor. Comput. Sci..

[79]  Lucian Ilie,et al.  Reducing NFAs by invariant equivalences , 2003, Theor. Comput. Sci..

[80]  Colin de la Higuera,et al.  A bibliographical study of grammatical inference , 2005, Pattern Recognit..

[81]  Thomas Schwentick,et al.  Existential second-order logic over graphs: Charting the tractability frontier , 2004, JACM.

[82]  Joachim Niehren,et al.  Learning n-Ary Node Selecting Tree Transducers from Completely Annotated Examples , 2006, ICGI.

[83]  Makoto Kanazawa,et al.  Identification in the limit of categorial grammars , 1993, J. Log. Lang. Inf..

[84]  Damián López,et al.  Characterizing Membrane Structures Through Multiset Tree Automata , 2007, Workshop on Membrane Computing.

[85]  W. Marsden.  I and J , 2012 .

[86]  Kaleem Siddiqi,et al.  Hamilton-Jacobi Skeletons , 2002, International Journal of Computer Vision.

[87]  James Rogers,et al.  Strict LT2 : Regular : : Local : Recognizable , 1996, LACL.

[88]  Ayumi Shinohara,et al.  Polynomial-time learning of elementary formal systems , 2000, New Generation Computing.

[89]  Ryo Yoshinaka,et al.  Distributional Learning of Simple Context-Free Tree Grammars , 2011, ALT.

[90]  Géraud Sénizergues,et al.  The Equivalence and Inclusion Problems for NTS Languages , 1985, J. Comput. Syst. Sci..

[91]  Ryo Yoshinaka,et al.  Efficient learning of multiple context-free languages with multidimensional substitutability from positive data , 2011, Theor. Comput. Sci..

[92]  Pedro García,et al.  IDENTIFYING REGULAR LANGUAGES IN POLYNOMIAL TIME , 1993 .

[93]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[94]  Tsunehiko Kameda,et al.  On the State Minimization of Nondeterministic Finite Automata , 1970, IEEE Transactions on Computers.

[95]  Barak A. Pearlmutter,et al.  Results of the Abbadingo One DFA Learning Competition and a New Evidence-Driven State Merging Algorithm , 1998, ICGI.

[96]  Barry Levine,et al.  Derivatives of Tree Sets with Applications to Grammatical Inference , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[97]  Anna Kasprzik,et al.  Inference of Residual Finite-State Tree Automata from Membership Queries and Finite Positive Data , 2011, Developments in Language Theory.

[98]  Aurélien Lemay,et al.  Learning Regular Languages Using Non Deterministic Finite Automata , 2000, ICGI.

[99]  Damián López,et al.  Error Correcting Analysis for Tree Languages , 2000, Int. J. Pattern Recognit. Artif. Intell..

[100]  David E. Muller,et al.  Alternating automata on infinite objects, determinacy and Rabin's theorem , 1984, Automata on Infinite Words.

[101]  Pedro García.  Learning k-Testable tree sets from positive data* , 2003 .

[102]  Yusuke Suzuki,et al.  Polynomial Time Inductive Inference of Cograph Pattern Languages from Positive Data , 2011, ILP.

[103]  J. Conway.  Regular algebra and finite machines , 1971 .

[104]  Aravind K. Joshi,et al.  Tree-Adjoining Grammars , 1997, Handbook of Formal Languages.

[105]  Menno van Zaanen.  ABL: Alignment-Based Learning , 2000, COLING.

[106]  Rani Siromoney,et al.  Learning of Recognizable Picture Languages , 1992, ICPIA.

[107]  Colin de la Higuera,et al.  Grammatical Inference: Learning Automata and Grammars , 2010 .

[108]  Damián López,et al.  Identifying P Rules from Membrane Structures with an Error-Correcting Approach , 2006, Workshop on Membrane Computing.

[109]  Kevin Knight,et al.  Training Tree Transducers , 2004, NAACL.

[110]  Manfred Droste,et al.  Weighted tree automata and weighted logics , 2006, Theor. Comput. Sci..

[111]  Enrique Vidal,et al.  Learning Locally Testable Languages in the Strict Sense , 1990, ALT.

[112]  Barry Levine.  The Use of Tree Derivatives and a Sample Support Parameter for Inferring Tree Systems , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[113]  Dana Angluin,et al.  Inductive Inference of Formal Languages from Positive Data , 1980, Inf. Control..

[114]  Werner Kuich.  Formal Power Series over Trees , 1997, Developments in Language Theory.

[115]  Anna Kasprzik,et al.  Making Finite-State Methods Applicable to Languages Beyond Context-Freeness via Multi-dimensional Trees , 2009, FSMNLP.

[116]  Christian Wurm,et al.  Completeness of Full Lambek Calculus for Syntactic Concept Lattices , 2013, FG.

[117]  Cristina Tîrnauca.  Correction Queries in Active Learning , 2010, Scientific Applications of Language Methods.

[118]  Edwin R. Hancock,et al.  Learning shape-classes using a mixture of tree-unions , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[119]  Stuart M. Shieber,et al.  Evidence against the context-freeness of natural language , 1985 .

[120]  David J. Weir,et al.  Characterizing Structural Descriptions Produced by Various Grammatical Formalisms , 1987, ACL.

[121]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[122]  Takeshi Shinohara,et al.  The correct definition of finite elasticity: corrigendum to identification of unions , 1991, COLT '91.

[123]  Ryo Yoshinaka,et al.  Towards Dual Approaches for Learning Context-Free Grammars Based on Syntactic Concept Lattices , 2011, Developments in Language Theory.

[124]  K. Kamata,et al.  Inference of tree automata from sample set of trees , 2004, International Journal of Computer & Information Sciences.

[125]  Yusuke Suzuki,et al.  Ordered term tree languages which are polynomial time inductively inferable from positive data , 2006, Theor. Comput. Sci..

[126]  Pedro García,et al.  Learning k-piecewise testable languages from positive data , 1996, ICGI.

[127]  Satoshi Kobayashi,et al.  Learning local languages and its application to protein /spl alpha/-chain identification , 1994, 1994 Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences.

[128]  Takayoshi Shoudai,et al.  Learning of Ordered Tree Languages with Height-Bounded Variables Using Queries , 2004, ALT.

[129]  H. Rulot,et al.  An efficient algorithm for the inference of circuit-free automata , 1988 .

[130]  R. Smullyan.  Theory of formal systems , 1962 .

[131]  Yasubumi Sakakibara,et al.  Pair hidden Markov models on tree structures , 2003, ISMB.

[132]  Olgierd Unold,et al.  Grammar-Based Classifier System for Recognition of Promoter Regions , 2007, ICANNGA.

[133]  Aurélien Lemay,et al.  Residual Finite State Automata , 2002, Fundam. Informaticae.

[134]  Edwin R. Hancock,et al.  Discovering Shape Categories by Clustering Shock Trees , 2001, CAIP.

[135]  Sandro Spina,et al.  Mutually Compatible and Incompatible Merges for the Search of the Smallest Consistent DFA , 2004, ICGI.

[136]  Edwin R. Hancock,et al.  Learning Structural Variations in Shock Trees , 2002, SSPR/SPR.

[137]  Edwin R. Hancock,et al.  Computing approximate tree edit distance using relaxation labeling , 2003, Pattern Recognit. Lett..

[138]  R. Durbin,et al.  Biological sequence analysis: Background on probability , 1998 .

[139]  Ryo Yoshinaka,et al.  Polynomial Time Learning of Some Multiple Context-Free Languages with a Minimally Adequate Teacher , 2010, FG.

[140]  Kaleem Siddiqi,et al.  Matching Hierarchical Structures Using Association Graphs , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[141]  Thomas Schwentick,et al.  Expressiveness and complexity of XML Schema , 2006, TODS.

[142]  Olgierd Unold,et al.  Use of Learning Classifier System for Inferring Natural Language Grammar , 2007, HIS.

[143]  Anna Kasprzik,et al.  Generalizing over Several Learning Settings , 2010, ICGI.

[144]  Nicholas Kushmerick,et al.  Wrapper Induction for Information Extraction , 1997, IJCAI.

[145]  Takayoshi Shoudai,et al.  A Polynomial Time Algorithm for Finding a Minimally Generalized Linear Interval Graph Pattern , 2009, IEICE Trans. Inf. Syst..

[146]  Henning Fernau,et al.  Proceedings Machines, Computations and Universality 2013: Formal Language Questions for Eulerian Trails , 2013 .

[147]  Aravind K. Joshi,et al.  Skeletal Structural Descriptions , 1978, Inf. Control..

[148]  Satoru Miyano,et al.  Inferring a Tree from Walks , 1996, Theor. Comput. Sci..

[149]  Yang Liu,et al.  Forest-to-String Statistical Translation Rules , 2007, ACL.

[150]  Colin de la Higuera,et al.  Inference of omega-Languages from Prefixes , 2001, ALT.

[151]  José M. Sempere,et al.  Learning Distinguishable Linear Grammars from Positive Data , 2004, ICGI.

[152]  Yasubumi Sakakibara,et al.  Learning context-free grammars from structural data in polynomial time , 1988, COLT '88.

[153]  Takayoshi Shoudai,et al.  Polynomial Time Algorithms for Finding Unordered Tree Patterns with Internal Variables , 2001, FCT.

[154]  Damián López,et al.  Inference of reversible tree languages , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[155]  Andreas Potthoff,et al.  Computing Small Nondeterministic Finite Automata , 1995 .

[156]  Henning Fernau,et al.  Learning XML Grammars , 2001, MLDM.

[157]  Yasubumi Sakakibara,et al.  Efficient Learning of Context-Free Grammars from Positive Structural Examples , 1992, Inf. Comput..

[158]  Andreas Maletti,et al.  Survey: Tree Transducers in Machine Translation , 2010, NCMA.

[159]  Yusuke Suzuki,et al.  Learning of Finite Unions of Tree Patterns with Internal Structured Variables from Queries , 2008, IEICE Trans. Inf. Syst..

[160]  Timo Knuutila.  Inference of k -testable Tree Languages , 1993 .

[161]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[162]  Damián López,et al.  Learning Decision Trees and Tree Automata for a Syntactic Pattern Recognition Task , 2003, IbPRIA.

[163]  R. Nussinov,et al.  Tree graphs of RNA secondary structures and their comparisons. , 1989, Computers and biomedical research, an international journal.

[164]  Alexander M. Rush,et al.  Induction of Probabilistic Synchronous Tree-Insertion Grammars , 2005 .

[165]  Makoto Kanazawa.  Learnable Classes of Categorial Grammars , 1998 .

[166]  Alessandro Moschitti,et al.  Fast and effective kernels for relational learning from texts , 2007, ICML '07.

[167]  Derick Wood,et al.  Standard Generalized Markup Language: Mathematical and Philosophical Issues , 1995, Computer Science Today.

[168]  H. Vogler,et al.  Weighted Tree Automata and Tree Transducers , 2009 .

[169]  Ryo Yoshinaka,et al.  Distributional Learning of Abstract Categorial Grammars , 2011, LACL.

[170]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[171]  Hiroki Arimura,et al.  Finding Tree Patterns Consistent with Positive and Negative Examples Using Queries , 1994, AII/ALT.

[172]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[173]  Damián López,et al.  Inference of k-Testable Directed Acyclic Graph Languages , 2012, ICGI.