The semantics of role labeling

The problem of ascribing a semantic representation to text is an important one that can help text understanding problems like textual entailment. In this thesis, we address the problem of assigning a shallow semantic representation to text. This problem is traditionally studied in the context of verbs and their nominalizations. We propose to extend the task to go beyond verbs and nominalizations to include other linguistic constructions such as commas and prepositions We develop an ontology of predicate-argument relations that commas and prepositions express in text. Just like the verb and nominal semantic role labeling schemes, the relations we propose are domain independent. For these two classes of phenomena, we introduce new corpora where these relations are annotated. From the machine learning perspective, learning to predicting these relations is a structured learning problem. However, we only have the small (for commas) or partially annotated (for prepositions) data sets. To predict the new relations, we show that using linguistic knowledge and information about output structure can bias the learning to build robust models. Finally, we observe that the relations expressed by the various phenomena interact with each other by constraining each others’ output. We show that we can take advantage of these interdependencies by enforcing coherence between their predictions. By constraining inference using linguistic knowledge, we can improve relation prediction performance.

[1]  Martha Palmer,et al.  PropBank: the Next Level of TreeBank , 2003 .

[2]  R. S. Jackendo,et al.  Toward an Explanatory Semantic Representation , 1976 .

[3]  Michael White,et al.  A More Precise Analysis of Punctuation for Broad-Coverage Surface Realization with CCG , 2008, COLING 2008.

[4]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[5]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[6]  Dirk Hovy,et al.  Disambiguation of Preposition Sense Using Linguistically Motivated Features , 2009, NAACL.

[7]  Cícero Nogueira dos Santos,et al.  Semantic Role Labeling , 2012 .

[8]  Ken Litkowski,et al.  The Preposition Project , 2021, ArXiv.

[9]  Jeffrey Gruber Studies in lexical relations , 1965 .

[10]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[11]  Christopher R. Johnson,et al.  Background to Framenet , 2003 .

[12]  Ken Litkowski Proposed Next Steps for The Preposition Project , 2012 .

[13]  Sebastian Riedel Improving the Accuracy and Efficiency of MAP Inference for Markov Logic , 2008, UAI.

[14]  Thorsten Joachims,et al.  Cutting-plane training of structural SVMs , 2009, Machine Learning.

[15]  Michael Collins,et al.  Head-Driven Statistical Models for Natural Language Parsing , 2003, CL.

[16]  Varol Akman,et al.  Current approaches to punctuation in computational linguistics , 1996, Comput. Humanit..

[17]  Oren Etzioni,et al.  Open Information Extraction from the Web , 2007, CACM.

[18]  Beth Levin,et al.  English Verb Classes and Alternations: A Preliminary Investigation , 1993 .

[19]  Xavier Carreras,et al.  Introduction to the CoNLL-2005 Shared Task: Semantic Role Labeling , 2005, CoNLL.

[20]  Gourab Kundu,et al.  Adapting Text instead of the Model: An Open Domain Approach , 2011, CoNLL.

[21]  Michael Collins,et al.  Three Generative, Lexicalised Models for Statistical Parsing , 1997, ACL.

[22]  Dan Roth,et al.  Modeling Discriminative Global Inference , 2007, International Conference on Semantic Computing (ICSC 2007).

[23]  Christine D. Doran,et al.  Incorporating Punctuation Into the Sentence Grammar: A Lexicalized Tree Adjoining Grammar Perspective , 1998 .

[24]  Eneko Agirre,et al.  Selectional Preferences for Semantic Role Classification , 2013, CL.

[25]  Timothy Baldwin,et al.  Prepositions in Applications: A Survey and Introduction to the Special Issue , 2009, CL.

[26]  Ralph Grishman,et al.  Message Understanding Conference- 6: A Brief History , 1996, COLING.

[27]  Ming-Wei Chang,et al.  Structured Output Learning with Indirect Supervision , 2010, ICML.

[28]  Alexander M. Rush,et al.  Exact Decoding of Syntactic Translation Models through Lagrangian Relaxation , 2011, ACL.

[29]  Eugene Charniak,et al.  Statistical Parsing with a Context-Free Grammar and Word Statistics , 1997, AAAI/IAAI.

[30]  Noah A. Smith,et al.  Probabilistic Frame-Semantic Parsing , 2010, NAACL.

[31]  Richard Johansson,et al.  The CoNLL-2009 Shared Task: Syntactic and Semantic Dependencies in Multiple Languages , 2009, CoNLL Shared Task.

[32]  Michael Collins,et al.  New Ranking Algorithms for Parsing and Tagging: Kernels over Discrete Structures, and the Voted Perceptron , 2002, ACL.

[33]  Dirk Hovy,et al.  What’s in a Preposition? Dimensions of Sense Disambiguation for an Interesting Word Class , 2010, COLING.

[34]  Toru Hirano,et al.  Detecting Semantic Relations between Named Entities in Text Using Contextual Features , 2007, ACL.

[35]  Daniel Jurafsky,et al.  Shallow Semantic Parsing using Support Vector Machines , 2004, NAACL.

[36]  Patrick Pantel,et al.  Espresso: Leveraging Generic Patterns for Automatically Harvesting Semantic Relations , 2006, ACL.

[37]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[38]  David M. Magerman Statistical Decision-Tree Models for Parsing , 1995, ACL.

[39]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[40]  Dan Roth,et al.  An NLP Curator (or: How I Learned to Stop Worrying and Love NLP Pipelines) , 2012, LREC.

[41]  Michael Collins,et al.  Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms , 2002, EMNLP.

[42]  Kenneth C. Litkowski,et al.  SemEval-2007 Task 06: Word-Sense Disambiguation of Prepositions , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[43]  Yoram Singer,et al.  Pegasos: primal estimated sub-gradient solver for SVM , 2011, Math. Program..

[44]  Eugene Charniak,et al.  Coarse-to-Fine n-Best Parsing and MaxEnt Discriminative Reranking , 2005, ACL.

[45]  Daniel Gildea,et al.  The Proposition Bank: An Annotated Corpus of Semantic Roles , 2005, CL.

[46]  Dan Roth,et al.  Knowledge Representation for Semantic Entailment and Question-Answering , 1995 .

[47]  Martha Palmer,et al.  Verbnet: a broad-coverage, comprehensive verb lexicon , 2005 .

[48]  Dan Roth,et al.  A Joint Model for Extended Semantic Role Labeling , 2011, EMNLP.

[49]  Jeffrey P. Bigham,et al.  Names and Similarities on the Web: Fact Extraction in the Fast Lane , 2006, ACL.

[50]  L. John Old An Analysis of Semantic Overlap among English Prepositions in Roget's Thesaurus. , 2003 .

[51]  Iván V. Meza,et al.  Jointly Identifying Predicates, Arguments and Senses using Markov Logic , 2009, NAACL.

[52]  Tommi S. Jaakkola,et al.  New Outer Bounds on the Marginal Polytope , 2007, NIPS.

[53]  Xavier Carreras,et al.  Introduction to the CoNLL-2004 Shared Task: Semantic Role Labeling , 2004, CoNLL.

[54]  David R. Dowty,et al.  Non-verbal thematic proto-roles. , 1993 .

[55]  Hwee Tou Ng,et al.  Semantic Role Labeling of NomBank: A Maximum Entropy Approach , 2006, EMNLP.

[56]  Katrin Erk,et al.  SemEval-2007 Task 19: Frame Semantic Structure Extraction , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[57]  Rashmi Prasad,et al.  The Penn Discourse Treebank , 2004, LREC.

[58]  Ralph Grishman,et al.  The NomBank Project: An Interim Report , 2004, FCP@NAACL-HLT.

[59]  Slav Petrov,et al.  A Universal Part-of-Speech Tagset , 2011, LREC.

[60]  Dan Roth,et al.  Design Challenges and Misconceptions in Named Entity Recognition , 2009, CoNLL.

[61]  Ann Bies,et al.  The Penn Treebank: Annotating Predicate Argument Structure , 1994, HLT.

[62]  Christopher D. Manning,et al.  Joint Parsing and Named Entity Recognition , 2009, NAACL.

[63]  Raymond J. Mooney,et al.  Learning to sportscast: a test of grounded language acquisition , 2008, ICML '08.

[64]  Michael Collins,et al.  Discriminative Reranking for Natural Language Parsing , 2000, CL.

[65]  David R. Dowty Thematic proto-roles and argument selection , 1991 .

[66]  Christopher D. Manning,et al.  Verb Sense and Subcategorization: Using Joint Inference to Improve Performance on Complementary Task , 2004, EMNLP.

[67]  Aravind K. Joshi,et al.  Tree-Adjoining Grammars , 1997, Handbook of Formal Languages.

[68]  Daniel Gildea,et al.  Automatic Labeling of Semantic Roles , 2000, ACL.

[69]  Andrew Y. Ng,et al.  Robust Textual Inference via Graph Matching , 2005, HLT.

[70]  Richard Johansson,et al.  The CoNLL 2008 Shared Task on Joint Parsing of Syntactic and Semantic Dependencies , 2008, CoNLL.

[71]  Dan Roth,et al.  Modeling Semantic Relations Expressed by Prepositions , 2013, TACL.

[72]  Thorsten Joachims,et al.  Learning structural SVMs with latent variables , 2009, ICML '09.

[73]  Ming-Wei Chang,et al.  Relation Alignment for Textual Entailment Recognition , 2009, TAC.

[74]  Doug Downey,et al.  Local and Global Algorithms for Disambiguation to Wikipedia , 2011, ACL.

[75]  Hwee Tou Ng,et al.  Joint Learning of Preposition Senses and Semantic Roles of Prepositional Phrases , 2009, EMNLP.

[76]  Luke S. Zettlemoyer,et al.  Learning Context-Dependent Mappings from Sentences to Logical Form , 2009, ACL.

[77]  Ari Rappoport,et al.  Fully Unsupervised Discovery of Concept-Specific Relationships by Web Mining , 2007, ACL.

[78]  Dekang Lin,et al.  Automatic Retrieval and Clustering of Similar Words , 1998, ACL.

[79]  Dan Roth,et al.  Extraction of Entailed Semantic Relations Through Syntax-Based Comma Resolution , 2008, ACL.

[80]  Christopher D. Manning,et al.  Joint Learning Improves Semantic Role Labeling , 2005, ACL.

[81]  Dan Klein,et al.  Coreference Resolution in a Modular, Entity-Centered Model , 2010, NAACL.

[82]  C. Fillmore FRAME SEMANTICS AND THE NATURE OF LANGUAGE * , 1976 .

[83]  Lucy Vanderwende,et al.  What Syntax Can Contribute in the Entailment Task , 2005, MLCW.

[84]  Mark Steedman,et al.  CCGbank: A Corpus of CCG Derivations and Dependency Structures Extracted from the Penn Treebank , 2007, CL.

[85]  Mihai Surdeanu,et al.  Combination Strategies for Semantic Role Labeling , 2007, J. Artif. Intell. Res..

[86]  Nianwen Xue,et al.  Calibrating Features for Semantic Role Labeling , 2004, EMNLP.

[87]  Thomas Hofmann,et al.  Large Margin Methods for Structured and Interdependent Output Variables , 2005, J. Mach. Learn. Res..

[88]  Dan Roth,et al.  Learning to Resolve Natural Language Ambiguities: A Unified Approach , 1998, AAAI/IAAI.

[89]  Dekang Lin,et al.  DIRT – Discovery of Inference Rules from Text , 2001 .

[90]  Mark A. Przybocki,et al.  The Automatic Content Extraction (ACE) Program – Tasks, Data, and Evaluation , 2004, LREC.

[91]  Alexander M. Rush,et al.  Dual Decomposition for Parsing with Non-Projective Head Automata , 2010, EMNLP.

[92]  Dan Roth,et al.  Generalized Inference with Multiple Semantic Role Labeling Systems , 2005, CoNLL.

[93]  Dan Roth,et al.  “Ask Not What Textual Entailment Can Do for You...” , 2010, ACL.

[94]  Varol Akman,et al.  An Analysis of English Punctuation , 1998 .

[95]  Martha Palmer,et al.  From TreeBank to PropBank , 2002, LREC.

[96]  L. Getoor,et al.  1 Global Inference for Entity and Relation Identification via a Linear Programming Formulation , 2007 .

[97]  Alexander M. Rush,et al.  On Dual Decomposition and Linear Programming Relaxations for Natural Language Processing , 2010, EMNLP.

[98]  Dan Roth,et al.  A Linear Programming Formulation for Global Inference in Natural Language Tasks , 2004, CoNLL.

[99]  Timothy Baldwin,et al.  Semantic role labeling of prepositional phrases , 2006, TALIP.

[100]  Ari Rappoport,et al.  Unsupervised Discovery of Generic Relationships Using Pattern Clusters and its Evaluation by Automatically Generated SAT Analogy Questions , 2008, ACL.

[101]  Yoav Goldberg,et al.  An Efficient Algorithm for Easy-First Non-Directional Dependency Parsing , 2010, NAACL.

[102]  Aron Culotta,et al.  Dependency Tree Kernels for Relation Extraction , 2004, ACL.

[103]  Timothy Baldwin,et al.  MELB-YB: Preposition Sense Disambiguation Using Rich Semantic Features , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[104]  Martha Palmer,et al.  Using prepositions to extend a verb lexicon , 2004, HLT-NAACL 2004.

[105]  Gennaro Chierchia,et al.  Meaning and Grammar: An Introduction to Semantics , 1990 .

[106]  Marvin Minsky,et al.  A framework for representing knowledge , 1974 .

[107]  Dan Roth,et al.  The Importance of Syntactic Parsing and Inference in Semantic Role Labeling , 2008, CL.

[108]  Sebastian van Delden,et al.  Combining finite state automata and a greedy learning algorithm to determine the syntactic roles of commas , 2002, 14th IEEE International Conference on Tools with Artificial Intelligence, 2002. (ICTAI 2002). Proceedings..

[109]  Ming-Wei Chang,et al.  Discriminative Learning over Constrained Latent Representations , 2010, NAACL.

[110]  Christopher D. Manning,et al.  A Global Joint Model for Semantic Role Labeling , 2008, CL.

[111]  Ido Dagan,et al.  The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[112]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[113]  Michael Collins,et al.  Exact Decoding of Phrase-Based Translation Models through Lagrangian Relaxation , 2011, EMNLP.

[114]  Dan Roth,et al.  Understanding the Value of Features for Coreference Resolution , 2008, EMNLP.

[115]  Dan Roth,et al.  Learning from natural instructions , 2011, Machine Learning.

[116]  Ming-Wei Chang,et al.  Structured learning with constrained conditional models , 2012, Machine Learning.

[117]  Ido Dagan,et al.  Semantic Inference at the Lexical-Syntactic Level for Textual Entailment Recognition , 2007, ACL-PASCAL@ACL.

[118]  Satoshi Sekine,et al.  On-Demand Information Extraction , 2006, ACL.

[119]  Dmitry Zelenko,et al.  Kernel Methods for Relation Extraction , 2002, J. Mach. Learn. Res..

[120]  Dan Roth,et al.  Semantic Role Labeling Via Integer Linear Programming Inference , 2004, COLING.

[121]  Martha Palmer,et al.  The Role of Semantic Roles in Disambiguating Verb Senses , 2005, ACL.

[122]  Josef Ruppenhofer,et al.  FrameNet II: Extended theory and practice , 2006 .

[123]  Gourab Kundu,et al.  On Amortizing Inference Cost for Structured Prediction , 2012, EMNLP.

[124]  Timothy Baldwin,et al.  Improving Parsing and PP Attachment Performance with Sense Information , 2008, ACL.

[125]  Yoav Freund,et al.  Large Margin Classification Using the Perceptron Algorithm , 1998, COLT' 98.