Dependency Grammar and Dependency Parsing

Despite a long and venerable tradition in descriptive linguistics, dependency grammar has until recently played a fairly marginal role both in theoretical linguistics and in natural language processing. The increasing interest in dependency-based representations in natural language parsing in recent years appears to be motivated both by the potential usefulness of bilexical relations in disambiguation and by the gains in efficiency that result from the more constrained parsing problem for these representations. In this paper, we will review the state of the art in dependency-based parsing, starting with the theoretical foundations of dependency grammar and moving on to consider both grammar-driven and data-driven methods for dependency parsing. We will limit our attention to systems for dependency parsing in a narrow sense, i.e. systems where the analysis assigned to an input sentence takes the form of a dependency structure. This means that we will not discuss systems that exploit dependency relations for the construction of another type of representation, such as the head-driven parsing models of Collins (1997, 1999). Moreover, we will restrict ourselves to systems for full parsing, which means that we will not deal with systems that produce a partial or underspecified representation of dependency structure, such as Constraint Grammar parsers (Karlsson, 1990; Karlsson et al., 1995).

[1]  Joakim Nivre,et al.  Incrementality in Deterministic Dependency Parsing , 2004 .

[2]  Joakim Nivre Constraints on Non-Projective Dependency Parsing , 2006, EACL.

[3]  Noam Chomsky,et al.  Remarks on Nominalization , 2020, Nominalization.

[4]  Anssi Yli-Jyrä,et al.  Multiplanarity - a model for dependency structures in treebanks , 2003 .

[5]  Fernando Pereira,et al.  Online Learning of Approximate Dependency Parsing Algorithms , 2006, EACL.

[6]  Dilek Z. Hakkani-Tür,et al.  Building a Turkish Treebank , 2003 .

[7]  Atanas Chanev,et al.  Portability of Dependency Parsing Algorithms An Application for Italian , 2005 .

[8]  Daniel Jurafsky,et al.  Learning Syntactic Patterns for Automatic Hypernym Discovery , 2004, NIPS.

[9]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[10]  Glenn Carroll,et al.  Two Experiments on Learning Probabilistic Dependency Grammars from Corpora , 1992 .

[11]  Jane J. Robinson Dependency Structures and Transformational Rules , 1970 .

[12]  Thorsten Brants,et al.  TnT – A Statistical Part-of-Speech Tagger , 2000, ANLP.

[13]  Robert E. Tarjan,et al.  Finding optimum branchings , 1977, Networks.

[14]  Igor Mel’čuk,et al.  Dependency Syntax: Theory and Practice , 1987 .

[15]  Michael Collins,et al.  Three Generative, Lexicalised Models for Statistical Parsing , 1997, ACL.

[16]  Ralph Debusmann,et al.  Topological Dependency Trees: A Constraint-Based Account of Linear Precedence , 2001, ACL.

[17]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[18]  Atro Voutilainen,et al.  A language-independent system for parsing unrestricted text , 1995 .

[19]  Jason Eisner,et al.  An Empirical Comparison of Probability Models for Dependency Grammar , 1997, ArXiv.

[20]  Christer Samuelsson,et al.  A Statistical Theory of Dependency Syntax , 2000, COLING.

[21]  Michael A. Covington,et al.  A Fundamental Algorithm for Dependency Parsing , 2004 .

[22]  Christopher D. Manning,et al.  The unsupervised learning of natural language structure , 2005 .

[23]  Brian Roark,et al.  Incremental Parsing with the Perceptron Algorithm , 2004, ACL.

[24]  Mark Johnson,et al.  Estimators for Stochastic “Unification-Based” Grammars , 1999, ACL.

[25]  Sabine Brants,et al.  The TIGER Treebank , 2001 .

[26]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[27]  Yuji Matsumoto,et al.  Machine Learning-based Dependency Analyzer for Chinese , 2005, J. Chin. Lang. Comput..

[28]  Alon Lavie,et al.  A Classifier-Based Parser with Linear Run-Time Complexity , 2005, IWPT.

[29]  Fernando Pereira,et al.  Discriminative learning and spanning tree algorithms for dependency parsing , 2006 .

[30]  Tsutomu Hirao,et al.  A Deterministic Word Dependency Analyzer Enhanced With Preference Learning , 2004, COLING.

[31]  Raymond J. Mooney,et al.  Learning Semantic Grammars with Constructive Inductive Logic Programming , 1993, AAAI.

[32]  Joakim Nivre,et al.  Deterministic Dependency Parsing of English Text , 2004, COLING.

[33]  Sebastian Riedel,et al.  The CoNLL 2007 Shared Task on Dependency Parsing , 2007, EMNLP.

[34]  Joakim Nivre,et al.  MAMBA Meets TIGER: Reconstructing a Swedish Treebank from Antiquity , 2005 .

[35]  Ruslan Mitkov,et al.  Shallow Language Processing Architecture for Bulgarian , 2002, COLING.

[36]  Ralph Debusmann,et al.  A declarative grammar formalism for dependency grammar , 2001 .

[37]  Daniel H. Younger,et al.  Recognition and Parsing of Context-Free Languages in Time n^3 , 1967, Inf. Control..

[38]  Andrzej Stachurski,et al.  Parallel Optimization: Theory, Algorithms and Applications , 2000, Parallel Distributed Comput. Pract..

[39]  Tadao Kasami,et al.  An Efficient Recognition and Syntax-Analysis Algorithm for Context-Free Languages , 1965 .

[40]  Alon Lavie,et al.  Parser Combination by Reparsing , 2006, NAACL.

[41]  Michael A. Covington,et al.  Discontinuous dependency parsing of free and fixed word order: Work in progress , 1994 .

[42]  Dekang Lin On the Structural Complexity of Natural Language Sentences , 1996, COLING.

[43]  Wen Wang,et al.  A Statistical Constraint Dependency Grammar (CDG) Parser , 2004 .

[44]  David Eppstein,et al.  Finding the k Smallest Spanning Trees , 1990, BIT.

[45]  David Milward,et al.  Dynamic dependency grammar , 1994 .

[46]  Richard Hudson,et al.  English word grammar , 1995 .

[47]  Jay Earley,et al.  An efficient context-free parsing algorithm , 1970, Commun. ACM.

[48]  Roberto Basili,et al.  Building the Italian Syntactic-Semantic Treebank , 2003 .

[49]  Alexis Nasr,et al.  A Simple String-Rewriting Formalism for Dependency Grammar , 2004, Workshop On Recent Advances In Dependency Grammar.

[50]  Ralph P. Grimaldi,et al.  Discrete and combinatorial mathematics , 1985 .

[51]  Michael A. Covington,et al.  A dependency parser for variable-word-order languages , 1990 .

[52]  Dmitry Zelenko,et al.  Kernel Methods for Relation Extraction , 2002, J. Mach. Learn. Res..

[53]  Eugene Charniak,et al.  Assigning Function Tags to Parsed Text , 2000, ANLP.

[54]  Robert C. Moore A Discriminative Framework for Bilingual Word Alignment , 2005, HLT.

[55]  Gertjan van Noord,et al.  The Alpino Dependency Treebank , 2001, CLIN.

[56]  Erhard W. Hinrichs,et al.  Is it Really that Difficult to Parse German? , 2006, EMNLP.

[57]  James R. Curran,et al.  The Importance of Supertagging for Wide-Coverage CCG Parsing , 2004, COLING.

[58]  Jan Hajic,et al.  Prague Arabic Dependency Treebank: Development in Data and Tools , 2004 .

[59]  Adam L. Berger,et al.  A Maximum Entropy Approach to Natural Language Processing , 1996, CL.

[60]  Satoshi Sekine,et al.  Automatic paraphrase acquisition from news articles , 2002 .

[61]  Francesco Maffioli,et al.  The k best spanning arborescences of a network , 1980, Networks.

[62]  M. Trautner,et al.  The Danish Dependency Treebank and the DTAG Treebank Tool , 2003 .

[63]  Wolfgang Menzel,et al.  Decision Procedures for Dependency Parsing Using Graded Constraints , 1998 .

[64]  Timo Järvinen,et al.  A non-projective dependency parser , 1997, ANLP.

[65]  Jason Eisner,et al.  Three New Probabilistic Models for Dependency Parsing: An Exploration , 1996, COLING.

[66]  Michael A. Covington Parsing Discontinuous constituents in Dependency Grammar , 1990, CL.

[67]  Kiril Ivanov Simov,et al.  Practical Annotation Scheme for an HPSG Treebank of Bulgarian , 2003, LINC@EACL.

[68]  P. Osenova,et al.  ‘An HPSG-based Syntactic Treebank of Bulgarian (BulTreeBank)’ , 2002 .

[69]  Yves Schabes,et al.  Parsing with Finite-State Transducers , 1997 .

[70]  Yuji Matsumoto,et al.  Japanese Dependency Analysis using Cascaded Chunking , 2002, CoNLL.

[71]  Koby Crammer,et al.  Online Passive-Aggressive Algorithms , 2003, J. Mach. Learn. Res..

[72]  Petya Osenova,et al.  Design and Implementation of the Bulgarian HPSG-based Treebank , 2004 .

[73]  Adwait Ratnaparkhi,et al.  A Maximum Entropy Model for Part-Of-Speech Tagging , 1996, EMNLP.

[74]  Yuji Matsumoto,et al.  Japanese Dependency Structure Analysis Based on Support Vector Machines , 2000, EMNLP.

[75]  Michael Collins,et al.  Head-Driven Statistical Models for Natural Language Parsing , 2003, CL.

[76]  Mary P. Harper,et al.  Extensions to constraint dependency parsing for spoken language processing , 1995, Comput. Speech Lang..

[77]  Haim Gaifman,et al.  Dependency Systems and Phrase-Structure Systems , 1965, Inf. Control..

[78]  Yuji Matsumoto MaltParser: A language-independent system for data-driven dependency parsing , 2005 .

[79]  Ralph Debusmann,et al.  Extensible Dependency Grammar: A New Methodology , 2004, Workshop On Recent Advances In Dependency Grammar.

[80]  Martin Kay,et al.  Syntactic Process , 1979, ACL.

[81]  Michael Collins,et al.  New Ranking Algorithms for Parsing and Tagging: Kernels over Discrete Structures, and the Voted Perceptron , 2002, ACL.

[82]  Lucien Tesnière Éléments de syntaxe structurale , 1959 .

[83]  姚小平,et al.  语言学简史 : [英文版] = A Short History of Linguistics , 1969 .

[84]  Ralph Grishman,et al.  The NomBank Project: An Interim Report , 2004, FCP@NAACL-HLT.

[85]  Fernando Pereira,et al.  Shallow Parsing with Conditional Random Fields , 2003, NAACL.

[86]  Martin Plátek,et al.  A Prototype of a Grammar Checker for Czech , 1997, ANLP.

[87]  D. G. Hays DEPENDENCY THEORY: A FORMALISM AND SOME OBSERVATIONS. MEM RM-4087-PR. , 1964, Memorandum RM-. Rand Corporation.

[88]  Wolfgang Menzel,et al.  A Transformation-based Parsing Technique With Anytime Properties , 2000, IWPT.

[89]  Michael A. Covington Syntactic Theory in the High Middle Ages: Subsequent developments , 1984 .

[90]  Adwait Ratnaparkhi,et al.  A Linear Observed Time Statistical Parser Based on Maximum Entropy Models , 1997, EMNLP.

[91]  Hiyan Alshawi,et al.  Head Automata and Bilingual Tiling: Translation with Minimal Representations , 1996, ACL.

[92]  Eugene Charniak,et al.  Coarse-to-Fine n-Best Parsing and MaxEnt Discriminative Reranking , 2005, ACL.

[93]  Michael Collins,et al.  A Statistical Parser for Czech , 1999, ACL.

[94]  Daniel Gildea,et al.  The Proposition Bank: An Annotated Corpus of Semantic Roles , 2005, CL.

[95]  Fred Karlsson,et al.  Constraint Grammar as a Framework for Parsing Running Text , 1990, COLING.

[96]  Michael Moortgat,et al.  Syntactic Analysis in the Spoken Dutch Corpus (CGN) , 2002, LREC.

[97]  Giorgio Satta,et al.  Analyzing an Italian Treebank with State-of-the-Art Statistical Parsers , 2004 .

[98]  Steven P. Abney Partial parsing via finite-state cascades , 1996, Natural Language Engineering.

[99]  Robert F. Simmons,et al.  The Acquisition and Use of Context-Dependent Grammars for English , 1992, Comput. Linguistics.

[100]  Srinivas Bangalore Localizing Dependencies and Supertagging , 2005 .

[101]  Dekang Lin,et al.  Dependency-Based Evaluation of Minipar , 2003 .

[102]  Aravind K. Joshi,et al.  Tree-Adjoining Grammars , 1997, Handbook of Formal Languages.

[103]  Joakim Nivre,et al.  Labeled Pseudo-Projective Dependency Parsing with Support Vector Machines , 2006, CoNLL.

[104]  Ann Bies,et al.  The Penn Treebank: Annotating Predicate Argument Structure , 1994, HLT.

[105]  James Henderson Inducing History Representations for Broad Coverage Statistical Parsing , 2003, HLT-NAACL.

[106]  Yuan Ding,et al.  Machine Translation Using Probabilistic Synchronous Dependency Insertion Grammars , 2005, ACL.

[107]  Erik F. Tjong Kim Sang,et al.  Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition , 2003, CoNLL.

[108]  Koby Crammer,et al.  Online Large-Margin Training of Dependency Parsers , 2005, ACL.

[109]  Andrew McCallum,et al.  Efficiently Inducing Features of Conditional Random Fields , 2002, UAI.

[110]  Ben Taskar,et al.  Learning structured prediction models: a large margin approach , 2005, ICML.

[111]  Vincenzo Lombardo,et al.  An Earley-type recognizer for dependency grammar , 1996, COLING.

[112]  Fernando Pereira,et al.  Multilingual Dependency Analysis with a Two-Stage Discriminative Parser , 2006, CoNLL.

[113]  Mark Johnson,et al.  Parsing the Wall Street Journal using a Lexical-Functional Grammar and Discriminative Estimation Techniques , 2002, ACL.

[114]  Eckhard Bick,et al.  Floresta Sintá(c)tica: A treebank for Portuguese , 2002, LREC.

[115]  Sabine Buchholz,et al.  CoNLL-X Shared Task on Multilingual Dependency Parsing , 2006, CoNLL.

[116]  Joakim Nivre,et al.  Memory-Based Dependency Parsing , 2004, CoNLL.

[117]  H. Hirakawa Semantic Dependency Analysis Method for Japanese based on Optimum Tree Search Algorithm Hideki Hirakawa , 2001 .

[118]  Frank Keller,et al.  Probabilistic Parsing for German Using Sister-Head Dependencies , 2003, ACL.

[119]  Ben Taskar,et al.  Max-Margin Parsing , 2004, EMNLP.

[120]  Roger Levy,et al.  Is it Harder to Parse Chinese, or the Chinese Treebank? , 2003, ACL.

[121]  Aron Culotta,et al.  Dependency Tree Kernels for Relation Extraction , 2004, ACL.

[122]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[123]  Ray Jackendoff,et al.  Semantic Interpretation in Generative Grammar , 1972 .

[124]  Ivan Titov,et al.  Porting Statistical Parsers with Data-Defined Kernels , 2006, CoNLL.

[125]  Keith Hall,et al.  Corrective Modeling for Non-Projective Dependency Parsing , 2005, IWPT.

[126]  Dekang Lin,et al.  A dependency-based method for evaluating broad-coverage parsers , 1995, Natural Language Engineering.

[127]  Timo Järvinen,et al.  Towards an implementable dependency grammar , 1998, ArXiv.

[128]  Brian Roark,et al.  Discriminative Language Modeling with Conditional Random Fields and the Perceptron Algorithm , 2004, ACL.

[129]  Ben Taskar,et al.  Max-Margin Markov Networks , 2003, NIPS.

[130]  Kemal Oflazer,et al.  The Annotation Process in the Turkish Treebank , 2003, LINC@EACL.

[131]  Montserrat Civit Torruella,et al.  Design Principles for a Spanish Treebank , 2002 .

[132]  Koby Crammer,et al.  Ultraconservative Online Algorithms for Multiclass Problems , 2001, J. Mach. Learn. Res..

[133]  菅山 謙正,et al.  Word Grammar 理論の研究 , 2005 .

[134]  Cristina Bosco,et al.  Treebank Development: the TUT Approach , 2002 .

[135]  Miguel Á. Carreira-Perpiñán,et al.  Multiscale conditional random fields for image labeling , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[136]  Atro Voutilainen Parsing Swedish , 2001, NODALIDA.

[137]  Joakim Nivre,et al.  Inductive Dependency Parsing of Natural Language Text , 2005 .

[138]  Matthias T. Kromann Optimality parsing and local cost functions in Discontinuous Grammar , 2004, FGMOL.

[139]  D. G. Hays Dependency Theory: A Formalism and Some Observations , 1964 .

[140]  Daniel Dominic Sleator,et al.  Parsing English with a Link Grammar , 1995, IWPT.

[141]  Walter Daelemans,et al.  MBT : Memory Based Tagger, version 1.0, Reference Guide , 2002 .

[142]  Chu-Ren Huang,et al.  Sinica Treebank: Design Criteria, Representational Issues and Implementation , 2004 .

[143]  David Chiang,et al.  Two Statistical Parsing Models Applied to the Chinese Treebank , 2000, ACL 2000.

[144]  Peter Hellwig,et al.  Dependency Unification Grammar , 1986, COLING.

[145]  Michael Collins,et al.  Discriminative Reranking for Natural Language Parsing , 2000, CL.

[146]  James R. Curran,et al.  Parsing the WSJ Using CCG and Log-Linear Models , 2004, ACL.

[147]  Stanley Starosta The case for lexicase : an outline of lexicase grammatical theory , 1989 .

[148]  Yuji Matsumoto,et al.  Statistical Dependency Analysis with Support Vector Machines , 2003, IWPT.

[149]  Vincenzo Lombardo,et al.  Integration of syntactic and lexical information in a hierarchical dependency grammar , 1998, Workshop On Processing Of Dependency-Based Grammars.

[150]  S. Marcus Sur la Notion de Projectivité , 1965 .

[151]  Alexis Nasr,et al.  Pseudo-Projectivity, A Polynomially Parsable Non-Projective Dependency Grammar , 1998, ACL.

[152]  Robert C. Berwick,et al.  The acquisition of syntactic knowledge , 1985 .

[153]  Fernando Pereira,et al.  Identifying gene and protein mentions in text using conditional random fields , 2005, BMC Bioinformatics.

[154]  Kemal Oflazer Dependency Parsing with an Extended Finite State Approach , 1999, ACL.

[155]  Martin Kay,et al.  Guides and Oracles for Linear-Time Parsing , 2000, IWPT.

[156]  David M. Magerman Statistical Decision-Tree Models for Parsing , 1995, ACL.

[157]  P. Resnik Treebanks : Building and Using Parsed Corpora , 2022 .

[158]  Michael Collins,et al.  Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms , 2002, EMNLP.

[159]  Jason Eisner,et al.  Bilexical Grammars and their Cubic-Time Parsing Algorithms , 2000 .

[160]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[161]  Aravind K. Joshi,et al.  Natural language parsing: Tree adjoining grammars: How much context-sensitivity is required to provide reasonable structural descriptions? , 1985 .

[162]  David R. Dowty On the Semantic Content of the Notion of ‘Thematic Role’ , 1989 .

[163]  John D. Lafferty,et al.  Towards History-based Grammars: Using Richer Models for Probabilistic Parsing , 1993, ACL.

[164]  Denys Duchier,et al.  Configuration of Labeled Trees under Lexicalized Constraints and Principles , 2003 .

[165]  Joakim Nivre,et al.  Inductive Dependency Parsing , 2006, Text, speech and language technology.

[166]  Yuji Matsumoto,et al.  Deterministic Dependency Structure Analyzer for Chinese , 2004, IJCNLP.

[167]  Joakim Nivre,et al.  Graph Transformations in Data-Driven Dependency Parsing , 2006, ACL.

[168]  Yuji Matsumoto,et al.  Chinese Deterministic Dependency Analyzer: Examining Effects of Global Features and Root Node Finder , 2005, SIGHAN@IJCNLP 2005.

[169]  Martha Palmer,et al.  Synchronous Dependency Insertion Grammars: A Grammar Formalism for Syntax Based Statistical MT , 2004 .

[170]  Cristina Bosco,et al.  A GRAMMATICAL RELATION SYSTEM FOR TREEBANK ANNOTATION , 2003 .

[171]  Kemal Oflazer,et al.  Statistical Dependency Parsing for Turkish , 2006, EACL.

[172]  Joakim Nivre,et al.  An Efficient Algorithm for Projective Dependency Parsing , 2003, IWPT.

[173]  Fernando Pereira,et al.  Non-Projective Dependency Parsing using Spanning Tree Algorithms , 2005, HLT.

[174]  Yoav Freund,et al.  Large Margin Classification Using the Perceptron Algorithm , 1998, COLT' 98.

[175]  Gertjan van Noord,et al.  Alpino: Wide-coverage Computational Analysis of Dutch , 2000, CLIN.

[176]  M. A. R T A P A L,et al.  The Penn Chinese TreeBank: Phrase structure annotation of a large corpus , 2005, Natural Language Engineering.

[177]  Joakim Nivre,et al.  A data-driven parser for Bulgarian , 2005 .

[178]  Walter Daelemans,et al.  A Memory-Based Alternative for Connectionist Shift-Reduce Parsing , 2000 .

[179]  Peter Neuhaus,et al.  The Complexity of Recognition of Linguistically Adequate Dependency Grammars , 1997, ACL.

[180]  Hiroshi Maruyama,et al.  Structural Disambiguation With Constraint Propagation , 1990, ACL.

[181]  Saso Dzeroski,et al.  Towards a Slovene Dependency Treebank , 2006, LREC.

[182]  Joakim Nivre,et al.  Pseudo-Projective Dependency Parsing , 2005, ACL.

[183]  Joakim Nivre,et al.  MaltParser: A Language-Independent System for Data-Driven Dependency Parsing , 2007, Natural Language Engineering.

[184]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[185]  Leonidas Georgiadis Arborescence optimization problems solvable by Edmonds' algorithm , 2003, Theor. Comput. Sci..

[186]  Adwait Ratnaparkhi,et al.  Learning to Parse Natural Language with Maximum Entropy Models , 1999, Machine Learning.

[187]  Eugene Charniak,et al.  A Maximum-Entropy-Inspired Parser , 2000, ANLP.

[188]  Wojciech Skut,et al.  An Annotation Scheme for Free Word Order Languages , 1997, ANLP.

[189]  C. Habel,et al.  Language , 1931, NeuroImage.

[190]  Ingo Schröder,et al.  Natural language parsing with graded constraints , 2002 .

[191]  Boon-Lock Yeo,et al.  Implementation issues in the development of the parsec parser , 1995, Softw. Pract. Exp..