ESBM: An Entity Summarization BenchMark

Entity summarization is the problem of computing an optimal compact summary for an entity by selecting a size-constrained subset of triples from RDF data. Entity summarization supports a multiplicity of applications and has led to fruitful research. However, there is a lack of evaluation efforts that cover the broad spectrum of existing systems. One reason is a lack of benchmarks for evaluation. Some benchmarks are no longer available, while others are small and have limitations. In this paper, we create an Entity Summarization BenchMark (ESBM) which overcomes the limitations of existing benchmarks and meets standard desiderata for a benchmark. Using this largest available benchmark for evaluating general-purpose entity summarizers, we perform the most extensive experiment to date where 9 existing systems are compared. Considering that all of these systems are unsupervised, we also implement and evaluate a supervised learning based system for reference.

[1]  Marcin Sydow,et al.  The notion of diversity in graphical entity summarisation on semantic knowledge graphs , 2013, Journal of Intelligent Information Systems.

[2]  Hamid R. Arabnia,et al.  Combining Word Embedding and Knowledge-Based Topic Modeling for Entity Summarization , 2018, 2018 IEEE 12th International Conference on Semantic Computing (ICSC).

[3]  Achim Rettinger,et al.  Browsing DBpedia Entities with Summaries , 2014, ESWC.

[4]  Ioan Toma,et al.  Leveraging Usage Data for Linked Data Movie Entity Summarization , 2012, ArXiv.

[5]  Susan Elliott Sim,et al.  Using benchmarking to advance research: a challenge to software engineering , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[6]  Amit P. Sheth,et al.  Gleaning Types for Literals in RDF Triples with Application to Entity Summarization , 2016, ESWC.

[7]  Andreas Thalhammer,et al.  Linked Data Entity Summarization , 2017 .

[8]  Harald Sack,et al.  Evaluating Entity Summarization Using a Game-Based Ground Truth , 2012, International Semantic Web Conference.

[9]  Gong Cheng,et al.  Entity Summarization: State of the Art and Future Challenges , 2019, ArXiv.

[10]  Stefanos D. Kollias,et al.  A String Metric for Ontology Alignment , 2005, SEMWEB.

[11]  Jens Lehmann,et al.  DBpedia - A large-scale, multilingual knowledge base extracted from Wikipedia , 2015, Semantic Web.

[12]  Mariano P. Consens,et al.  Linked Movie Data Base , 2009, LDOW.

[13]  Kalpa Gunaratna.  Semantics-based Summarization of Entities in Knowledge Graphs , 2017 .

[14]  Yuzhong Qu,et al.  C3D+P: A summarization method for interactive entity resolution , 2015, J. Web Semant..

[15]  Krisztian Balog,et al.  Dynamic Factual Summaries for Entity Cards , 2017, SIGIR.

[17]  Harald Sack,et al.  FRanCo - A Ground Truth Corpus for Fact Ranking Evaluation , 2015, SumPre-HSWI@ESWC.

[18]  Karl Aberer,et al.  Contextualized ranking of entity types based on knowledge graphs , 2016, J. Web Semant..

[20]  Amit P. Sheth,et al.  FACES: Diversity-Aware Entity Summarization Using Incremental Hierarchical Conceptual Clustering , 2015, AAAI.

[21]  Yuzhong Qu,et al.  RELIN: Relatedness and Informativeness-Based Centrality for Entity Summarization , 2011, International Semantic Web Conference.

[22]  Hamid R. Arabnia,et al.  ES-LDA: Entity Summarization using Knowledge-based Topic Modeling , 2017, IJCNLP.

[23]  Zhibing Liu,et al.  MPSUM: Entity Summarization with Predicate-based Matching , 2020, ArXiv.

[24]  Nelia Lasierra,et al.  LinkSUM: Using Link Analysis to Summarize Entity Data , 2016, ICWE.

[25]  Simon Gottschalk,et al.  EventKG - the Hub of Event Knowledge on the Web - and Biographical Timeline Generation , 2019, Semantic Web.

[26]  Gjergji Kasneci,et al.  Assigning global relevance scores to DBpedia facts , 2014, 2014 IEEE 30th International Conference on Data Engineering Workshops.

[27]  Yuzhong Qu,et al.  Summarizing Entity Descriptions for Effective and Efficient Human-centered Entity Linking , 2015, WWW.

[28]  Yuzhong Qu,et al.  CD at ENSEC 2016: Generating Characteristic and Diverse Entity Summaries , 2016, SumPre@ESWC.