Distributional learning of parallel multiple context-free grammars

Natural languages require grammars beyond context-free for their description. Here we extend a family of distributional learning algorithms for context-free grammars to the class of Parallel Multiple Context-Free Grammars (pmcfgs). These grammars have two additional operations beyond the simple context-free operation of concatenation: the ability to interleave strings of symbols, and the ability to copy or duplicate strings. This allows the grammars to generate some non-semilinear languages, which are outside the class of mildly context-sensitive grammars. These grammars, if augmented with a suitable feature mechanism, are capable of representing all of the syntactic phenomena that have been claimed to exist in natural language.We present a learning algorithm for a large subclass of these grammars, that includes all regular languages but not all context-free languages. This algorithm relies on a generalisation of the notion of distribution as a function from tuples of strings to entire sentences; we define nonterminals using finite sets of these functions. Our learning algorithm uses a nonprobabilistic learning paradigm which allows for membership queries as well as positive samples; it runs in polynomial time.

[1]  Dana Angluin,et al.  Inference of Reversible Languages , 1982, JACM.

[2]  Ryo Yoshinaka,et al.  Efficient learning of multiple context-free languages with multidimensional substitutability from positive data , 2011, Theor. Comput. Sci..

[3]  A. Andrews.  Semantic case‐stacking and inside‐out unification∗ , 1996 .

[4]  Alexander Okhotin,et al.  Conjunctive Grammars , 2001, J. Autom. Lang. Comb..

[5]  Sharon Inkelas,et al.  Reduplication: Abbreviations used in morpheme glosses , 2005 .

[6]  Ryo Yoshinaka,et al.  Polynomial-Time Identification of Multiple Context-Free Languages from Positive Data and Membership Queries , 2010, ICGI.

[7]  Raymond M. Smullyan,et al.  Theory of Formal Systems. (AM-47) , 1961 .

[8]  Rajesh Bhatt,et al.  Semilinearity Is a Syntactic Invariant: A Reply to Michaelis and Kracht 1997 , 2004, Linguistic Inquiry.

[9]  R. Smullyan.  Theory of formal systems , 1962 .

[10]  Stuart M. Shieber,et al.  Foundational issues in natural language processing , 1991 .

[11]  Grzegorz Rozenberg,et al.  Developments in Language Theory II , 2002 .

[12]  Alexander Clark,et al.  Polynomial Identification in the Limit of Substitutable Context-free Languages , 2005 .

[13]  Peter Ljunglöf,et al.  A Polynomial Time Extension of Parallel Multiple Context-Free Grammar , 2005, LACL.

[14]  Sharon Inkelas,et al.  Reduplication: Index of subjects , 2005 .

[15]  Nicholas D. Evans,et al.  A grammar of Kayardild : with historical-comparative notes on Tangkic , 1995 .

[16]  David J. Weir,et al.  The equivalence of four extensions of context-free grammars , 1994, Mathematical systems theory.

[17]  Dana Angluin,et al.  Finding Patterns Common to a Set of Strings , 1980, J. Comput. Syst. Sci..

[18]  Daniel Radzinski,et al.  Chinese Number-Names, Tree Adjoining Languages, and Mild Context-Sensitivity , 1991, Comput. Linguistics.

[19]  Shalom Lappin,et al.  Linguistic Nativism and the Poverty of the Stimulus , 2011 .

[20]  Sharon Inkelas,et al.  The dual theory of reduplication , 2008 .

[21]  Jeffrey Heinz,et al.  Bounded copying is subsequential: Implications for metathesis and reduplication , 2012, SIGMORPHON.

[22]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[23]  Noam Chomsky,et al.  Poverty of the Stimulus Revisited , 2011, Cogn. Sci..

[24]  Takeshi Shinohara,et al.  Rich Classes Inferable from Positive Data: Length-Bounded Elementary Formal Systems , 1994, Inf. Comput..

[25]  David J. Weir,et al.  The convergence of mildly context-sensitive grammar formalisms , 1990 .

[26]  Marcus Kracht,et al.  Interpreted Languages and Compositionality , 2011 .

[27]  Annius Groenink.  Literal Movement Grammars , 1995, EACL.

[28]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[29]  Pierre Boullier,et al.  Chinese Numbers, MIX, Scrambling, and Range Concatenation Grammars , 1999, EACL.

[30]  Stuart M. Shieber,et al.  Evidence against the context-freeness of natural language , 1985 .

[31]  David J. Weir,et al.  Characterizing Structural Descriptions Produced by Various Grammatical Formalisms , 1987, ACL.

[32]  Ryo Yoshinaka,et al.  Towards Dual Approaches for Learning Context-Free Grammars Based on Syntactic Concept Lattices , 2011, Developments in Language Theory.

[33]  José M. Sempere,et al.  Grammatical inference: theoretical results and applications : 10th international colloquium ; proceedings , 2010 .

[34]  Alaa A. Kharbouch,et al.  Three models for the description of language , 1956, IRE Trans. Inf. Theory.

[35]  Leonor Becerra-Bonache,et al.  Inferring Grammars for Mildly Context Sensitive Languages in Polynomial-Time , 2006, ICGI.

[36]  Annius Groenink,et al.  Mild Context-Sensitivity and Tuple-Based Generalizations of Context-Grammar , 1996 .

[37]  Los Angeles,et al.  Generating Copies: An investigation into structural identity in language and grammar , 2006 .

[38]  Stephen Pulman,et al.  Generalised Phrase Structure Grammar‚ Earley's Algorithm‚ and the Minimisation of Recursion , 1983 .

[39]  Louisa Sadler,et al.  Case stacking in realizational morphology , 2006 .

[40]  Tadao Kasami,et al.  On Multiple Context-Free Grammars , 1991, Theor. Comput. Sci..

[41]  Ryo Yoshinaka,et al.  Polynomial Time Learning of Some Multiple Context-Free Languages with a Minimally Adequate Teacher , 2010, FG.

[42]  Marcus Kracht,et al.  Semilinearity as a Syntactic Invariant , 1996, LACL.

[43]  Sharon Inkelas,et al.  Reduplication: Doubling in Morphology , 2005 .

[44]  Alexander Clark,et al.  Learning Context Free Grammars with the Syntactic Concept Lattice , 2010, ICGI.