Bootstrapping Syntax and Recursion using Alginment-Based Learning

This paper introduces a new type of unsupervised learning algorithm, based on the alignment of sentences and Harris’s (1951) notion of interchangeability. The algorithm is applied to an untagged, unstructured corpus of natural language sentences, resulting in a labelled, bracketed version of the corpus. Firstly, the algorithm aligns all sentences in the corpus in pairs, resulting in a partition of the sentences consisting of parts of the sentences that are similar in both sentences and parts that are dissimilar. This information is used to find (possibly overlapping) constituents. Next, the algorithm selects (nonoverlapping) constituents. Several instances of the algorithm are applied to the ATIS corpus (Marcus et al., 1993) and the OVIS corpus (Bonnema et al., 1997). Apart from the promising numerical results, the most striking result is that even the simplest algorithm based on alignment learns recursion.

[1]  Khalil Sima'an,et al.  Learning Efficient Disambiguation , 1999, ArXiv.

[2]  Nick Chater,et al.  Distributional Information: A Powerful Cue for Acquiring Syntactic Categories , 1998, Cogn. Sci..

[3]  Mitchell P. Marcus,et al.  Parsing a Natural Language Using Mutual Information Statistics , 1990, AAAI.

[4]  Walter Daelemans,et al.  Memory-based lexical acquisition and processing , 1993, EAMT.

[5]  Van Nostrand,et al.  Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm , 1967 .

[6]  Carl de Marcken,et al.  Unsupervised language acquisition , 1996, ArXiv.

[7]  Peter Grünwald,et al.  A minimum description length approach to grammar inference , 1995, Learning for Natural Language Processing.

[8]  J. Gerard Wolff,et al.  Language acquisition, data compression and generalization , 1982 .

[9]  Eugene Charniak,et al.  Figures of Merit for Best-First Probabilistic Chart Parsing , 1998, Comput. Linguistics.

[10]  Steve Young,et al.  Applications of stochastic context-free grammars using the Inside-Outside algorithm , 1990 .

[11]  Rens Bod,et al.  Beyond Grammar: An Experience-Based Theory of Language , 1998 .

[12]  J. Baker.  Trainable grammars for speech recognition , 1979 .

[13]  Andreas Stolcke,et al.  Inducing Probabilistic Grammars by Bayesian Model Merging , 1994, ICGI.

[14]  Stanley F. Chen,et al.  Bayesian Grammar Induction for Language Modeling , 1995, ACL.

[15]  Taylor L. Booth,et al.  Probabilistic Representation of Formal Languages , 1969, SWAT.

[16]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[17]  Fernando Pereira,et al.  Inside-Outside Reestimation From Partially Bracketed Corpora , 1992, HLT.

[18]  C. M. Cook,et al.  Grammatical inference by hill climbing , 1976, Inf. Sci..

[19]  Z. Harris,et al.  Methods in structural linguistics. , 1952 .

[20]  Rens Bod,et al.  A DOP Model for Semantic Interpretation , 1997, ACL.

[21]  Michael J. Fischer,et al.  The String-to-String Correction Problem , 1974, JACM.

[22]  Eric Brill,et al.  Automatic Grammar Induction and Parsing Free Text: A Transformation-Based Approach , 1993, ACL.