Framing Named Entity Linking Error Types

Named Entity Linking (NEL) and relation extraction forms the backbone of Knowledge Base Population tasks. The recent rise of large open source Knowledge Bases and the continuous focus on improving NEL performance has led to the creation of automated benchmark solutions during the last decade. The benchmarking of NEL systems offers a valuable approach to understand a NEL system’s performance quantitatively. However, an in-depth qualitative analysis that helps improving NEL methods by identifying error causes usually requires a more thorough error analysis. This paper proposes a taxonomy to frame common errors and applies this taxonomy in a survey study to assess the performance of four well-known Named Entity Linking systems on three recent gold standards. Keywords: Named Entity Linking, Linked Data Quality, Corpora, Evaluation, Error Analysis

[1]  Heng Ji,et al.  Overview of TAC-KBP2016 Tri-lingual EDL and Its Impact on End-to-End KBP , 2016, TAC.

[2]  Gerhard Weikum,et al.  Robust Disambiguation of Named Entities in Text , 2011, EMNLP.

[3]  Heiko Paulheim,et al.  Evaluating Entity Linking: An Analysis of Current Benchmark Datasets and a Roadmap for Doing a Better Job , 2016, LREC.

[4]  Benjamin Heinzerling,et al.  Visual Error Analysis for Entity Linking , 2015, ACL.

[5]  Jens Lehmann,et al.  NIF Combinator: Combining NLP Tool Output , 2012, EKAW.

[6]  Raphaël Troncy,et al.  GERBIL: General Entity Annotator Benchmarking Framework , 2015, WWW.

[7]  Joel Nothman,et al.  Naïve but effective NIL clustering baselines - CMCRC at TAC 2011 , 2011, TAC.

[8]  Axel-Cyrille Ngonga Ngomo,et al.  All that Glitters Is Not Gold - Rule-Based Curation of Reference Datasets for Named Entity Recognition and Entity Linking , 2017, ESWC.

[9]  Pablo N. Mendes,et al.  Improving efficiency and accuracy in multilingual entity extraction , 2013, I-SEMANTICS '13.

[10]  J. Fleiss.  Measuring nominal scale agreement among many raters. , 1971 .

[11]  Massimiliano Ciaramita,et al.  A framework for benchmarking entity-annotation systems , 2013, WWW.

[12]  Arno Scharl,et al.  A Regional News Corpora for Contextualized Entity Discovery and Linking , 2016, LREC.

[13]  Arno Scharl,et al.  Consolidating Heterogeneous Enterprise Data for Named Entity Linking and Web Intelligence , 2015, Int. J. Artif. Intell. Tools.

[14]  Gerhard Weikum,et al.  KORE: keyphrase overlap relatedness for entity disambiguation , 2012, CIKM.

[15]  Joel Nothman,et al.  Cheap and easy entity evaluation , 2014, ACL.

[16]  Will Radford.  Linking named entities to Wikipedia , 2014 .

[17]  Roberto Navigli,et al.  Entity Linking meets Word Sense Disambiguation: a Unified Approach , 2014, TACL.

[18]  Sebastian Hellmann,et al.  N³ - A Collection of Datasets for Named Entity Recognition and Disambiguation in the NLP Interchange Format , 2014, LREC.