A Bayesian Approach to Unsupervised Semantic Role Induction

We introduce two Bayesian models for unsupervised semantic role labeling (SRL) task. The models treat SRL as clustering of syntactic signatures of arguments with clusters corresponding to semantic roles. The first model induces these clusterings independently for each predicate, exploiting the Chinese Restaurant Process (CRP) as a prior. In a more refined hierarchical model, we inject the intuition that the clusterings are similar across different predicates, even though they are not necessarily identical. This intuition is encoded as a distance-dependent CRP with a distance between two syntactic signatures indicating how likely they are to correspond to a single semantic role. These distances are automatically induced within the model and shared across predicates. Both models achieve state-of-the-art results when evaluated on PropBank, with the coupled model consistently outperforming the factored counterpart in all experimental set-ups.

[1]  Mirella Lapata,et al.  Unsupervised Semantic Role Induction via Split-Merge Clustering , 2011, ACL.

[2]  Robert L. Mercer,et al.  Class-Based n-gram Models of Natural Language , 1992, CL.

[3]  Mirella Lapata,et al.  Graph Alignment for Semi-Supervised Semantic Role Labeling , 2009, EMNLP.

[4]  Sebastian Riedel,et al.  The CoNLL 2007 Shared Task on Dependency Parsing , 2007, EMNLP.

[5]  Dan Klein,et al.  Learning Dependency-Based Compositional Semantics , 2011, CL.

[6]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[7]  Wayne H. Ward,et al.  Towards Robust Semantic Role Labeling , 2007, CL.

[8]  Emin Orhan.  Dirichlet Processes , 2012 .

[9]  Stephan Vogel,et al.  Corpus Expansion for Statistical Machine Translation with Semantic Role Label Substitution Rules , 2011, ACL.

[10]  Oren Etzioni,et al.  Open Information Extraction from the Web , 2007, CACM.

[11]  Christopher D. Manning,et al.  Spectral Chinese Restaurant Processes: Nonparametric Clustering Based on Similarities , 2011, AISTATS.

[12]  Dekang Lin,et al.  DIRT – Discovery of Inference Rules from Text , 2001 .

[13]  Dan Klein,et al.  Learning Semantic Correspondences with Less Supervision , 2009, ACL.

[14]  Marie-Francine Moens,et al.  Semi-supervised Semantic Role Labeling Using the Latent Words Language Model , 2009, EMNLP.

[15]  Daniel Jurafsky,et al.  Automatic Labeling of Semantic Roles , 2002, CL.

[16]  Mirella Lapata,et al.  Cross-lingual Annotation Projection for Semantic Roles , 2009, J. Artif. Intell. Res..

[17]  Daniel Gildea,et al.  The Proposition Bank: An Annotated Corpus of Semantic Roles , 2005, CL.

[18]  Mirella Lapata,et al.  Unsupervised Induction of Semantic Roles , 2010, HLT-NAACL.

[19]  Mirella Lapata,et al.  Unsupervised Semantic Role Induction with Graph Partitioning , 2011, EMNLP.

[20]  Ding Liu,et al.  Semantic Role Features for Machine Translation , 2010, COLING.

[21]  Suzanne Stevenson,et al.  Unsupervised Semantic Role Labellin , 2004, EMNLP.

[22]  Lonneke van der Plas,et al.  Scaling up Automatic Cross-Lingual Semantic Role Annotation , 2011, ACL.

[23]  Yoshua Bengio,et al.  Word Representations: A Simple and General Method for Semi-Supervised Learning , 2010, ACL.

[24]  Caroline Sporleder,et al.  Evaluating FrameNet-style semantic parsing: the role of coverage gaps in FrameNet , 2010, COLING.

[25]  Patrick Pantel,et al.  DIRT @SBT@discovery of inference rules from text , 2001, KDD '01.

[26]  Christopher D. Manning,et al.  Unsupervised Discovery of a Statistical Verb Lexicon , 2006, EMNLP.

[27]  Dan Roth,et al.  Confidence Driven Unsupervised Semantic Parsing , 2011, ACL.

[28]  Richard Johansson,et al.  The CoNLL-2009 Shared Task: Syntactic and Semantic Dependencies in Multiple Languages , 2009, CoNLL Shared Task.

[29]  Hoifung Poon,et al.  Unsupervised Semantic Parsing , 2009, EMNLP.

[30]  Ivan Titov,et al.  A Bayesian Model for Unsupervised Semantic Parsing , 2011, ACL.

[31]  Beth Levin,et al.  English Verb Classes and Alternations: A Preliminary Investigation , 1993 .

[32]  Ming-Wei Chang,et al.  Relation Alignment for Textual Entailment Recognition , 2009, TAC.

[33]  Bonnie Lynn Webber,et al.  Question Answering based on Semantic Roles , 2007, ACL 2007.

[34]  Roberto Basili,et al.  Cross-Language Frame Semantics Transfer in Bilingual Corpora , 2009, CICLing.

[35]  Ivan Titov,et al.  Bootstrapping Semantic Analyzers from Non-Contradictory Texts , 2010, ACL.

[36]  Pascale Fung,et al.  Semantic Roles for SMT: A Hybrid Two-Pass Model , 2009, NAACL.

[37]  Xavier Carreras,et al.  Introduction to the CoNLL-2005 Shared Task: Semantic Role Labeling , 2005, CoNLL.

[38]  Richard Johansson,et al.  The CoNLL 2008 Shared Task on Joint Parsing of Syntactic and Semantic Dependencies , 2008, CoNLL.

[39]  Hal Daumé,et al.  Fast search for Dirichlet process mixture models , 2007, AISTATS.

[40]  Jason D. M. Rennie.  Improving multi-class text classification with Naive Bayes , 2001 .

[41]  Jason A. Duan,et al.  Generalized spatial dirichlet process models , 2007 .

[42]  Peter I. Frazier,et al.  Distance dependent Chinese restaurant processes , 2009, ICML.

[43]  Ari Rappoport,et al.  Unsupervised Argument Identification for Semantic Role Labeling , 2009, ACL.

[44]  Mirella Lapata,et al.  Using Semantic Roles to Improve Question Answering , 2007, EMNLP.

[45]  Rohit J. Kate,et al.  Learning Language Semantics from Ambiguous Supervision , 2007, AAAI.