Integrating Selectional Constraints and Subcategorization Frames in a Dependency Parser

Statistical parsers are trained on treebanks that are composed of a few thousand sentences. In order to prevent data sparseness and computational complexity, such parsers make strong independence hypotheses on the decisions that are made to build a syntactic tree. These independence hypotheses yield a decomposition of the syntactic structures into small pieces, which in turn prevent the parser from adequately modeling many lexico-syntactic phenomena like selectional constraints and subcategorization frames. Additionally, treebanks are several orders of magnitude too small to observe many lexico-syntactic regularities, such as selectional constraints and subcategorization frames. In this article, we propose a solution to both problems: how to account for patterns that exceed the size of the pieces that are modeled in the parser and how to obtain subcategorization frames and selectional constraints from raw corpora and incorporate them in the parsing process. The method proposed was evaluated on French and on English. The experiments on French showed a decrease of 41.6% of selectional constraint violations and a decrease of 22% of erroneous subcategorization frame assignment. These figures are lower for English: 16.21% in the first case and 8.83% in the second.

[1]  David Chiang,et al.  Forest Rescoring: Faster Decoding with Integrated Language Models , 2007, ACL.

[2]  Hermann Ney,et al.  Confidence measures for large vocabulary continuous speech recognition , 2001, IEEE Trans. Speech Audio Process..

[3]  Eric P. Xing,et al.  Concise Integer Linear Programming Formulations for Dependency Parsing , 2009, ACL.

[4]  Bernd Bohnet,et al.  Top Accuracy and Fast Dependency Parsing is not a Contradiction , 2010, COLING.

[5]  Noah A. Smith,et al.  Turning on the Turbo: Fast Third-Order Non-Projective Turbo Parsers , 2013, ACL.

[6]  Liang Huang,et al.  Forest Reranking: Discriminative Parsing with Non-Local Features , 2008, ACL.

[7]  Rebecca Hwa,et al.  Sample Selection for Statistical Parsing , 2004, CL.

[8]  Keith Hall,et al.  Corrective Modeling for Non-Projective Dependency Parsing , 2005, IWPT.

[9]  Cédric Messiant,et al.  A Subcategorization Acquisition System for French Verbs , 2008, ACL.

[10]  Pascal Denis,et al.  Analyse syntaxique du français : des constituants aux dépendances , 2009 .

[11]  Mary P. Harper,et al.  Self-Training with Products of Latent Variable Grammars , 2010, EMNLP.

[12]  Christopher D. Manning Automatic Acquisition of a Large Sub Categorization Dictionary From Corpora , 1993, ACL.

[13]  Martin Volk,et al.  Exploiting the WWW as a corpus to resolve PP attachment ambiguities , 2001 .

[14]  Alexis Nasr,et al.  Constructing parse forests that include exactly the n-best PCFG trees , 2009, IWPT.

[15]  Jason Eisner,et al.  Bilexical Grammars and their Cubic-Time Parsing Algorithms , 2000 .

[16]  Alexandra Kinyon,et al.  Building a Treebank for French , 2000, LREC.

[17]  Aravind K. Joshi,et al.  Natural language parsing: Tree adjoining grammars: How much context-sensitivity is required to provide reasonable structural descriptions? , 1985 .

[18]  Michael Collins,et al.  Efficient Third-Order Dependency Parsers , 2010, ACL.

[19]  Alexis Nasr,et al.  Enforcing Subcategorization Constraints in a Parser Using Sub-parses Recombining , 2013, HLT-NAACL.

[20]  Eugene Charniak,et al.  Coarse-to-Fine n-Best Parsing and MaxEnt Discriminative Reranking , 2005, ACL.

[21]  Koby Crammer,et al.  Online Large-Margin Training of Dependency Parsers , 2005, ACL.

[22]  Li Cai,et al.  Exploiting Web-Derived Selectional Preference to Improve Statistical Dependency Parsing , 2011, ACL.

[23]  David Chiang,et al.  Better k-best Parsing , 2005, IWPT.

[24]  Noah A. Smith,et al.  Dependency Parsing , 2009, Encyclopedia of Artificial Intelligence.

[25]  Marie Candito,et al.  Parse Correction with Specialized Models for Difficult Attachment Types , 2011, EMNLP.

[26]  José-Miguel Benedí,et al.  Statistical Confidence Measures for Probabilistic Parsing , 2009, RANLP.

[27]  Joseph Le Roux,et al.  Semi-supervised Dependency Parsing using Lexical Affinities , 2012, ACL.

[28]  Benjamin Van Durme,et al.  Annotated Gigaword , 2012, AKBC-WEKEX@NAACL-HLT.

[29]  Giuseppe Attardi,et al.  Tree Revision Learning for Dependency Parsing , 2007, NAACL.

[30]  Michael Collins,et al.  Discriminative Reranking for Natural Language Parsing , 2000, CL.

[31]  Kenneth Ward Church,et al.  Using Web-scale N-grams to Improve Base NP Parsing Performance , 2010, COLING.

[32]  Sebastian Riedel,et al.  Incremental Integer Linear Programming for Non-projective Dependency Parsing , 2006, EMNLP.

[33]  Philipp Koehn,et al.  Synthesis Lectures on Human Language Technologies , 2016 .

[34]  Alexander M. Rush,et al.  Vine Pruning for Efficient Multi-Pass Dependency Parsing , 2012, NAACL.

[35]  Jason Eisner,et al.  Three New Probabilistic Models for Dependency Parsing: An Exploration , 1996, COLING.

[36]  Daniel Gildea,et al.  Corpus Variation and Parser Performance , 2001, EMNLP.

[37]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[38]  Frédéric Béchet,et al.  Robust dependency parsing for spoken language understanding of spontaneous speech , 2009, INTERSPEECH.

[39]  Michael R. Brent,et al.  Automatic Acquisition of Subcategorization Frames from Tagged Text , 1991, HLT.

[40]  Giorgio Satta,et al.  On the Complexity of Non-Projective Data-Driven Dependency Parsing , 2007, IWPT.

[41]  Michael Collins,et al.  Three Generative, Lexicalised Models for Statistical Parsing , 1997, ACL.

[42]  Daniel M. Bikel,et al.  Intricacies of Collins’ Parsing Model , 2004, CL.

[43]  Fernando Pereira,et al.  Online Learning of Approximate Dependency Parsing Algorithms , 2006, EACL.

[44]  Kenneth Ward Church,et al.  Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[45]  Frédéric Béchet,et al.  MACAON An NLP Tool Suite for Processing Word Lattices , 2011, ACL.

[46]  Tobias Achterberg,et al.  SCIP: solving constraint integer programs , 2009, Math. Program. Comput..

[47]  Richard M. Karp,et al.  Reducibility Among Combinatorial Problems , 1972, 50 Years of Integer Programming.

[48]  Dan Klein,et al.  Web-Scale Features for Full-Scale Parsing , 2011, ACL.

[49]  Preslav Nakov,et al.  Using the Web as an Implicit Training Set: Application to Structural Ambiguity Resolution , 2005, HLT.

[50]  Thierry Poibeau,et al.  LexSchem: a Large Subcategorization Lexicon for French Verbs , 2008, LREC.

[51]  Alexis Nasr,et al.  Active Learning for Dependency Parsing Using Partially Annotated Sentences , 2011, IWPT.

[52]  Bernd Bohnet,et al.  Very high accuracy and fast dependency parsing is not a contradiction , 2010, COLING 2010.