Prune-and-Score: Learning for Greedy Coreference Resolution

We propose a novel search-based approach for greedy coreference resolution, where the mentions are processed in order and added to previous coreference clusters. Our method is distinguished by the use of two functions to make each coreference decision: a pruning function that prunes bad coreference decisions from further consideration, and a scoring function that then selects the best among the remaining decisions. Our framework reduces learning of these functions to rank learning, which helps leverage powerful off-the-shelf rank-learners. We show that our Prune-and-Score approach is superior to using a single scoring function to make both decisions and outperforms several state-of-the-art approaches on multiple benchmark corpora including OntoNotes.

[1]  Lynette Hirschman,et al.  A Model-Theoretic Coreference Scoring Scheme , 1995, MUC.

[2]  Breck Baldwin,et al.  Algorithms for Scoring Coreference Chains , 1998 .

[3]  Hwee Tou Ng,et al.  A Machine Learning Approach to Coreference Resolution of Noun Phrases , 2001, CL.

[4]  Claire Gardent,et al.  Improving Machine Learning Approaches to Coreference Resolution , 2002, ACL.

[5]  Andrew McCallum,et al.  Toward Conditional Models of Identity Uncertainty with Application to Proper Noun Coreference , 2003, IIWeb.

[6]  Roni Khardon,et al.  Learning to Take Actions , 1996, Machine Learning.

[7]  Yuji Matsumoto MaltParser: A language-independent system for data-driven dependency parsing , 2005 .

[8]  Michael Collins,et al.  Discriminative Reranking for Natural Language Parsing , 2000, CL.

[9]  William W. Cohen,et al.  Stacked Sequential Learning , 2005, IJCAI.

[10]  Daniel Marcu,et al.  Learning as search optimization: approximate large margin methods for structured prediction , 2005, ICML.

[11]  Thorsten Joachims,et al.  Supervised clustering with support vector machines , 2005, ICML.

[12]  Xiaoqiang Luo,et al.  On Coreference Resolution Performance Metrics , 2005, HLT.

[13]  Daniel Marcu,et al.  Practical structured learning techniques for natural language processing , 2006 .

[14]  David A. McAllester,et al.  The Generalized A* Architecture , 2007, J. Artif. Intell. Res..

[15]  Andrew McCallum,et al.  First-Order Probabilistic Models for Coreference Resolution , 2007, NAACL.

[16]  Dan Roth,et al.  Understanding the Value of Features for Coreference Resolution , 2008, EMNLP.

[17]  Pascal Denis,et al.  Specialized Models and Ranking for Coreference Resolution , 2008, EMNLP.

[18]  John Langford,et al.  Search-based structured prediction , 2009, Machine Learning.

[19]  Thorsten Joachims,et al.  Learning structural SVMs with latent variables , 2009, ICML '09.

[20]  Alan Fern,et al.  Learning Linear Ranking Functions for Beam Search with Application to Planning , 2009, J. Mach. Learn. Res..

[21]  Claire Cardie,et al.  Coreference Resolution with Reconcile , 2010, ACL.

[22]  Christopher J. C. Burges,et al.  From RankNet to LambdaRank to LambdaMART: An Overview , 2010 .

[23]  Ben Taskar,et al.  Structured Prediction Cascades , 2010, AISTATS.

[24]  Tie-Yan Liu,et al.  Learning to rank for information retrieval , 2009, SIGIR.

[25]  Dan Klein,et al.  Coreference Resolution in a Modular, Entity-Centered Model , 2010, NAACL.

[26]  Vincent Ng,et al.  Coreference Resolution with World Knowledge , 2011, ACL.

[27]  Vincent Ng,et al.  Narrowing the Modeling Gap: A Cluster-Ranking Approach to Coreference Resolution , 2014, J. Artif. Intell. Res..

[28]  Andrew McCallum,et al.  SampleRank: Training Factor Graphs with Atomic Gradients , 2011, ICML.

[29]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[30]  Dan Roth,et al.  Illinois-Coref: The UI System in the CoNLL-2012 Shared Task , 2012, EMNLP-CoNLL Shared Task.

[31]  Veselin Stoyanov,et al.  Easy-first Coreference Resolution , 2012, COLING.

[32]  Eraldo Rezende Fernandes,et al.  Latent Structure Perceptron with Feature Induction for Unrestricted Coreference Resolution , 2012, EMNLP-CoNLL Shared Task.

[33]  Dan Roth,et al.  Learning-based Multi-Sieve Co-reference Resolution with Knowledge , 2012, EMNLP-CoNLL.

[34]  Andrew McCallum,et al.  A Discriminative Hierarchical Model for Fast Coreference at Large Scale , 2012, ACL.

[35]  Yuchen Zhang,et al.  CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes , 2012, EMNLP-CoNLL Shared Task.

[36]  Andrew McCallum,et al.  Dynamic Knowledge-Base Alignment for Coreference Resolution , 2013, CoNLL.

[37]  Dan Klein,et al.  Easy Victories and Uphill Battles in Coreference Resolution , 2013, EMNLP.

[38]  Thomas G. Dietterich,et al.  Learning to Detect Basal Tubules of Nematocysts in SEM Images , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[39]  Heng Ji,et al.  Joint Event Extraction via Structured Prediction with Global Features , 2013, ACL.

[40]  Dan Klein,et al.  Decentralized Entity-Level Modeling for Coreference Resolution , 2013, ACL.

[41]  Dan Roth,et al.  A Constrained Latent Variable Model for Coreference Resolution , 2013, EMNLP.

[42]  Luke S. Zettlemoyer,et al.  Joint Coreference Resolution and Named-Entity Linking with Multi-Pass Sieves , 2013, EMNLP.

[43]  Heeyoung Lee,et al.  Deterministic Coreference Resolution Based on Entity-Centric, Precision-Ranked Rules , 2013, CL.

[44]  Joakim Nivre,et al.  Training Deterministic Parsers with Non-Deterministic Oracles , 2013, TACL.

[45]  Alan Fern,et al.  Structured prediction via output space search , 2014, J. Mach. Learn. Res..

[46]  Alan Fern,et al.  Multi-object Tracking via Constrained Sequential Labeling , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Alan Fern,et al.  HC-Search: A Learning Framework for Search-based Structured Prediction , 2014, J. Artif. Intell. Res..

[48]  Jonas Kuhn,et al.  Learning Structured Perceptrons for Coreference Resolution with Latent Antecedents and Non-local Features , 2014, ACL.

[49]  Jun Yu,et al.  HC-Search for Multi-Label Prediction: An Empirical Study , 2014, AAAI.