A Constrained Latent Variable Model for Coreference Resolution

Coreference resolution is a well known clustering task in Natural Language Processing. In this paper, we describe the Latent Left Linking model (L 3 M), a novel, principled, and linguistically motivated latent structured prediction approach to coreference resolution. We show that L 3 M admits efficient inference and can be augmented with knowledge-based constraints; we also present a fast stochastic gradient based learning. Experiments on ACE and Ontonotes data show that L 3 M and its constrained version, CL 3 M, are more accurate than several state-of-the-art approaches as well as some structured prediction models proposed in the literature.

[1]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[2]  Lynette Hirschman,et al.  A Model-Theoretic Coreference Scoring Scheme , 1995, MUC.

[3]  Breck Baldwin,et al.  Algorithms for Scoring Coreference Chains , 1998 .

[4]  Hwee Tou Ng,et al.  A Machine Learning Approach to Coreference Resolution of Noun Phrases , 2001, CL.

[5]  Claire Gardent,et al.  Improving Machine Learning Approaches to Coreference Resolution , 2002, ACL.

[6]  Andrew McCallum,et al.  Toward Conditional Models of Identity Uncertainty with Application to Proper Noun Coreference , 2003, IIWeb.

[7]  Alan L. Yuille,et al.  The Concave-Convex Procedure , 2003, Neural Computation.

[8]  Dan Roth,et al.  A Linear Programming Formulation for Global Inference in Natural Language Tasks , 2004, CoNLL.

[9]  Vincent Ng Supervised Ranking for Pronoun Resolution: Some Recent Improvements , 2005, AAAI.

[10]  Thorsten Joachims,et al.  Supervised clustering with support vector machines , 2005, ICML.

[11]  Xiaoqiang Luo,et al.  On Coreference Resolution Performance Metrics , 2005, HLT.

[12]  Mirella Lapata,et al.  Constraint-Based Sentence Compression: An Integer Programming Approach , 2006, ACL.

[13]  Pascal Denis,et al.  Joint Determination of Anaphoricity and Coreference Resolution using Integer Programming , 2007, NAACL.

[14]  Andrew McCallum,et al.  First-Order Probabilistic Models for Coreference Resolution , 2007, NAACL.

[15]  Rada Mihalcea,et al.  Wikify!: linking documents to encyclopedic knowledge , 2007, CIKM '07.

[16]  Dan Roth,et al.  Understanding the Value of Features for Coreference Resolution , 2008, EMNLP.

[17]  Pascal Denis,et al.  Specialized Models and Ranking for Coreference Resolution , 2008, EMNLP.

[18]  Claire Cardie,et al.  Conundrums in Noun Phrase Coreference Resolution: Making Sense of the State-of-the-Art , 2009, ACL.

[19]  Thorsten Joachims,et al.  Learning structural SVMs with latent variables , 2009, ICML '09.

[20]  Pascal Denis,et al.  Global joint models for coreference resolution and named entity classification , 2009, Proces. del Leng. Natural.

[21]  Noah A. Smith,et al.  Softmax-Margin CRFs: Training Log-Linear Models with Cost Functions , 2010, NAACL.

[22]  Heeyoung Lee,et al.  A Multi-Pass Sieve for Coreference Resolution , 2010, EMNLP.

[23]  Joan A. Smith,et al.  Robust , Light-weight Approaches to compute Lexical Similarity , 2010 .

[24]  Dan Klein,et al.  Coreference Resolution in a Modular, Entity-Centered Model , 2010, NAACL.

[25]  Joachim M. Buhmann,et al.  Entropy and Margin Maximization for Structured Output Learning , 2010, ECML/PKDD.

[26]  Dan Roth,et al.  Robust, Light-weight Approaches to compute Lexical Similarity , 2010 .

[27]  Anthony Wirth,et al.  Correlation Clustering , 2010, Encyclopedia of Machine Learning and Data Mining.

[28]  Ben Taskar,et al.  Posterior Regularization for Structured Latent Variable Models , 2010, J. Mach. Learn. Res..

[29]  Alexander M. Rush,et al.  Dual Decomposition for Parsing with Non-Projective Head Automata , 2010, EMNLP.

[30]  Vincent Ng,et al.  Coreference Resolution with World Knowledge , 2011, ACL.

[31]  Vincent Ng,et al.  Narrowing the Modeling Gap: A Cluster-Ranking Approach to Coreference Resolution , 2014, J. Artif. Intell. Res..

[32]  Dan Roth,et al.  Inference Protocols for Coreference Resolution , 2011, CoNLL Shared Task.

[33]  Heeyoung Lee,et al.  Stanford’s Multi-Pass Sieve Coreference Resolution System at the CoNLL-2011 Shared Task , 2011, CoNLL Shared Task.

[34]  Heng Ji,et al.  Knowledge Base Population: Successful Approaches and Challenges , 2011, ACL.

[35]  Vincent Ng,et al.  Ensemble-Based Coreference Resolution , 2011, IJCAI.

[36]  Nianwen Xue,et al.  CoNLL-2011 Shared Task: Modeling Unrestricted Coreference in OntoNotes , 2011, CoNLL Shared Task.

[37]  Doug Downey,et al.  Local and Global Algorithms for Disambiguation to Wikipedia , 2011, ACL.

[38]  Dan Roth,et al.  Illinois-Coref: The UI System in the CoNLL-2012 Shared Task , 2012, EMNLP-CoNLL Shared Task.

[39]  Chen Chen,et al.  Combining the Best of Two Worlds: A Hybrid Approach to Multilingual Coreference Resolution , 2012, EMNLP-CoNLL Shared Task.

[40]  Veselin Stoyanov,et al.  Easy-first Coreference Resolution , 2012, COLING.

[41]  Eraldo Rezende Fernandes,et al.  Latent Structure Perceptron with Feature Induction for Unrestricted Coreference Resolution , 2012, EMNLP-CoNLL Shared Task.

[42]  Dan Klein,et al.  Coreference Semantics from Web Features , 2012, ACL.

[43]  Ming-Wei Chang,et al.  Structured learning with constrained conditional models , 2012, Machine Learning.

[44]  Dan Roth,et al.  Learning-based Multi-Sieve Co-reference Resolution with Knowledge , 2012, EMNLP-CoNLL.

[45]  Richárd Farkas,et al.  Data-driven Multilingual Coreference Resolution using Resolver Stacking , 2012, EMNLP-CoNLL Shared Task.

[46]  Ming-Wei Chang,et al.  Unified Expectation Maximization , 2012, NAACL.

[47]  Yuchen Zhang,et al.  CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes , 2012, EMNLP-CoNLL Shared Task.

[48]  Michael Strube,et al.  A Multigraph Model for Coreference Resolution , 2012, EMNLP-CoNLL Shared Task.

[49]  Dan Klein,et al.  Decentralized Entity-Level Modeling for Coreference Resolution , 2013, ACL.

[50]  Heeyoung Lee,et al.  Deterministic Coreference Resolution Based on Entity-Centric, Precision-Ranked Rules , 2013, CL.