Using Universal Linguistic Knowledge to Guide Grammar Induction

We present an approach to grammar induction that utilizes syntactic universals to improve dependency parsing across a range of languages. Our method uses a single set of manually-specified language-independent rules that identify syntactic dependencies between pairs of syntactic categories that commonly occur across languages. During inference of the probabilistic model, we use posterior expectation constraints to require that a minimum proportion of the dependencies we infer be instances of these rules. We also automatically refine the syntactic categories given in our coarsely tagged input. Across six languages our approach outperforms state-of-the-art unsupervised methods by a significant margin.

[1]  Kathleen R. McKeown,et al.  Predicting the semantic orientation of adjectives , 1997 .

[2]  Bart de Boer,et al.  The Atoms of Language: The Mind's Hidden Rules of Grammar; Foundations of Language: Brain, Meaning, Grammar, Evolution , 2002, Artificial Life.

[3]  Oren Etzioni,et al.  Extracting Product Features and Opinions from Reviews , 2005, HLT.

[4]  Mark Johnson,et al.  Improving Unsupervised Dependency Parsing with Richer Contexts and Smoothing , 2009, NAACL.

[5]  Dan Klein,et al.  Learning from measurements in exponential families , 2009, ICML '09.

[6]  Janyce Wiebe,et al.  Learning Subjective Adjectives from Corpora , 2000, AAAI/IAAI.

[7]  Razvan C. Bunescu,et al.  A Shortest Path Dependency Kernel for Relation Extraction , 2005, HLT.

[8]  Sabine Buchholz,et al.  CoNLL-X Shared Task on Multilingual Dependency Parsing , 2006, CoNLL.

[9]  Gideon S. Mann,et al.  Semi-supervised Learning of Dependency Parsers using Generalized Expectation Criteria , 2009, ACL/IJCNLP.

[10]  Philip J. Stone,et al.  Extracting Information. (Book Reviews: The General Inquirer. A Computer Approach to Content Analysis) , 1967 .

[11]  Sabine Bergler,et al.  Mining WordNet for a Fuzzy Sentiment: Sentiment Tag Extraction from WordNet Glosses , 2006, EACL.

[12]  Foster Provost,et al.  Machine Learning from Imbalanced Data Sets 101 , 2008 .

[13]  Dan Klein,et al.  The Infinite PCFG Using Hierarchical Dirichlet Processes , 2007, EMNLP.

[14]  Michael Collins,et al.  Head-Driven Statistical Models for Natural Language Parsing , 2003, CL.

[15]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[16]  Dan Klein,et al.  Corpus-Based Induction of Syntactic Structure: Models of Dependency and Constituency , 2004, ACL.

[17]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[18]  Dan Klein,et al.  Learning and Inference for Hierarchically Split PCFGs , 2007, AAAI.

[19]  Janyce Wiebe,et al.  Effects of Adjective Orientation and Gradability on Sentence Subjectivity , 2000, COLING.

[20]  Swapna Somasundaran,et al.  Detecting Arguing and Sentiment in Meetings , 2007, SIGdial.

[21]  Janyce Wiebe,et al.  Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis , 2005, HLT.

[22]  Xiaoyan Zhu,et al.  Using Conditional Random Fields to Extract Contexts and Answers of Questions from Online Forums , 2008, ACL.

[23]  Ben Taskar,et al.  Posterior Regularization for Structured Latent Variable Models , 2010, J. Mach. Learn. Res..

[24]  Noah A. Smith,et al.  Variational Inference for Grammar Induction with Prior Knowledge , 2009, ACL/IJCNLP.

[25]  Hong Yu,et al.  Towards Answering Opinion Questions: Separating Facts from Opinions and Identifying the Polarity of Opinion Sentences , 2003, EMNLP.

[26]  Takashi Inui,et al.  Extracting Semantic Orientations of Words using Spin Model , 2005, ACL.

[27]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[28]  M. de Rijke,et al.  UvA-DARE ( Digital Academic Repository ) Using WordNet to measure semantic orientations of adjectives , 2004 .

[29]  Michael L. Littman,et al.  Measuring praise and criticism: Inference of semantic orientation from association , 2003, TOIS.

[30]  Hal Daumé,et al.  A Bayesian Model for Discovering Typological Implications , 2007, ACL.

[31]  Young-In Song,et al.  Finding question-answer pairs from online forums , 2008, SIGIR '08.

[32]  Mark C. Baker.  The Atoms of Language , 1987 .

[33]  Emily M. Bender.  Linguistically Naïve != Language Independent: Why NLP Needs Linguistic Typology , 2009 .

[34]  Jeonghee Yi,et al.  Sentiment analysis: capturing favorability using natural language processing , 2003, K-CAP '03.

[35]  Ming Zhou,et al.  Extracting Chatbot Knowledge from Online Discussion Forums , 2007, IJCAI.

[36]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[37]  Ben Taskar,et al.  Dependency Grammar Induction via Bitext Projection Constraints , 2009, ACL/IJCNLP.

[38]  Jure Leskovec,et al.  Predicting positive and negative links in online social networks , 2010, WWW '10.

[39]  Hiroshi Kanayama,et al.  Fully Automatic Lexicon Expansion for Domain-oriented Sentiment Analysis , 2006, EMNLP.

[40]  Regina Barzilay,et al.  Unsupervised Multilingual Grammar Induction , 2009, ACL.

[41]  Christian Bauckhage,et al.  The slashdot zoo: mining a social network with negative edges , 2009, WWW.

[42]  Soo-Min Kim,et al.  Determining the Sentiment of Opinions , 2004, COLING.

[43]  Chen Lin,et al.  Simultaneously modeling semantics and structure of threaded discussions: a sparse coding approach and its applications , 2009, SIGIR.

[44]  Noah A. Smith,et al.  Shared Logistic Normal Distributions for Soft Parameter Tying in Unsupervised Grammar Induction , 2009, NAACL.

[45]  Kevin R. Gregg.  SECOND LANGUAGE ACQUISITION AND UNIVERSAL GRAMMAR , 2004, Studies in Second Language Acquisition.

[46]  Dan Klein,et al.  Prototype-Driven Grammar Induction , 2006, ACL.

[47]  Dan Klein,et al.  Accurate Unlexicalized Parsing , 2003, ACL.

[48]  Ellen Riloff,et al.  Learning Extraction Patterns for Subjective Expressions , 2003, EMNLP.

[49]  Christopher D. Manning,et al.  The Infinite Tree , 2007, ACL.

[50]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[51]  Dan Klein,et al.  Phylogenetic Grammar Induction , 2010, ACL.

[52]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[53]  Jonas Kuhn.  Experiments in parallel-text based grammar induction , 2004, ACL.

[54]  Swapna Somasundaran,et al.  Finding the Sources and Targets of Subjective Expressions , 2008, LREC.

[55]  Ben Taskar,et al.  Expectation Maximization and Posterior Constraints , 2007, NIPS.

[56]  Qiang Yang,et al.  Thread detection in dynamic text message streams , 2006, SIGIR.

[57]  Ming-Wei Chang,et al.  Guiding Semi-Supervision with Constraint-Driven Learning , 2007, ACL.

[58]  Dan Klein,et al.  Two Languages are Better than One (for Syntactic Parsing) , 2008, EMNLP.

[59]  Rada Mihalcea,et al.  A Bootstrapping Method for Building Subjectivity Lexicons for Languages with Scarce Resources , 2008, LREC.

[60]  F. Newmeyer.  Possible and probable languages: A generative perspective on linguistic typology , 2008 .

[61]  Satoshi Morinaga,et al.  Mining product reputations on the Web , 2002, KDD.

[62]  Matt Thomas,et al.  Get out the vote: Determining support or opposition from Congressional floor-debate transcripts , 2006, EMNLP.

[63]  Ben Taskar,et al.  Posterior vs Parameter Sparsity in Latent Variable Models , 2009, NIPS.

[64]  Claire Cardie,et al.  Annotating Expressions of Opinions and Emotions in Language , 2005, Lang. Resour. Evaluation.