Benchmarking RDF Query Engines and Instance Matching Systems

Standards and benchmarking have traditionally been used as the main tools to formally define and provably illustrate the level of the adequacy of systems to address the new challenges. In this chapter, we discuss benchmarks for RDF query engines and instance matching systems. In practice, benchmarks are used to inform users of the strengths and weaknesses of competing tools and approaches, but more importantly, they encourage the advancement of technology by providing both academia and industry with clear targets for performance and functionality.

[1]  Heiko Stoermer,et al.  Results of OKKAM Feature based Entity Matching Algorithm for Instance Matching Contest of OAEI 2009 , 2009, OM.

[2]  Axel-Cyrille Ngonga Ngomo,et al.  BorderFlow: A Local Graph Clustering Algorithm for Natural Language Processing , 2009, CICLing.

[3]  Chen Li,et al.  Supporting Efficient Record Linkage for Large Data Sets Using Mapping Techniques , 2006, World Wide Web.

[4]  Gerhard Weikum,et al.  WWW 2007 / Track: Semantic Web Session: Ontologies ABSTRACT YAGO: A Core of Semantic Knowledge , 2022 .

[5]  Robert Isele,et al.  Learning Expressive Linkage Rules using Genetic Programming , 2012, Proc. VLDB Endow..

[6]  Atanas Kiryakov,et al.  Benchmarking RDF Query Engines: The LDBC Semantic Publishing Benchmark , 2016, BLINK@ISWC.

[7]  Robert Isele,et al.  Active learning of expressive linkage rules using genetic programming , 2013, J. Web Semant..

[8]  Heiner Stuckenschmidt,et al.  Benchmarking Matching Applications on the Semantic Web , 2011, ESWC.

[9]  Mansur R. Kabuka,et al.  ASMOV Results for OAEI 2007 , 2007, OM.

[10]  Muhammad Saleem,et al.  FEASIBLE: A Feature-Based SPARQL Benchmark Generation Framework , 2015, SEMWEB.

[11]  Gerhard Weikum,et al.  LINDA: distributed web-of-data-scale entity matching , 2012, CIKM.

[12]  Stephan Bloehdorn,et al.  The SWRC Ontology - Semantic Web for Research Communities , 2005, EPIA.

[13]  Heiner Stuckenschmidt,et al.  Results of the Ontology Alignment Evaluation Initiative , 2007 .

[14]  Cosmin Stroe,et al.  Using AgreementMaker to align ontologies for OAEI 2010 , 2010, OM.

[15]  Éric Gaussier,et al.  A Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation , 2005, ECIR.

[16]  Thomas Neumann,et al.  TPC-H Analyzed: Hidden Messages and Lessons Learned from an Influential Benchmark , 2013, TPCTC.

[17]  Abderrahmane Khiat,et al.  STRIM results for OAEI 2015 instance matching evaluation , 2015, OM.

[18]  Silvana Castano,et al.  Semantic Information Interoperability in Open Networked Systems , 2004, ICSNW.

[19]  Jens Lehmann,et al.  DBpedia SPARQL Benchmark - Performance Assessment with Real Queries on Real Data , 2011, SEMWEB.

[20]  Alfio Ferrara,et al.  Towards a Benchmark for Instance Matching , 2008, OM.

[21]  Gerhard Weikum,et al.  RDF-3X: a RISC-style engine for RDF , 2008, Proc. VLDB Endow..

[22]  Feng Shi,et al.  RiMOM Results for OAEI 2009 , 2008, OM.

[23]  M. Tamer Özsu,et al.  Diversified Stress Testing of RDF Data Management Systems , 2014, SEMWEB.

[24]  Jan Nößner,et al.  CODI: Combinatorial Optimization for Data Integration: results for OAEI 2011 , 2010, OM.

[25]  Mehrnoush Shamsfard,et al.  SBUEI: results for OAEI 2012 , 2012, OM.

[26]  Ian Horrocks,et al.  LogMap and LogMapLt results for OAEI 2012 , 2012, OM.

[27]  Reinhold Weicker,et al.  An overview of common benchmarks , 1990, Computer.

[28]  Pável Calado,et al.  An Overview of XML Duplicate Detection Algorithms , 2010, Soft Computing in XML Data Management.

[29]  Salvatore J. Stolfo,et al.  The merge/purge problem for large databases , 1995, SIGMOD '95.

[30]  M. Tamer Özsu,et al.  XBench benchmark and performance testing of XML DBMSs , 2004, Proceedings. 20th International Conference on Data Engineering.

[31]  Haofen Wang,et al.  Zhishi.links results for OAEI 2011 , 2011, OM.

[32]  Katrin Simone Zaiß,et al.  Instance-based ontology matching and the evaluation of matching systems , 2010 .

[33]  Panagiotis G. Ipeirotis,et al.  Duplicate Record Detection: A Survey , 2007 .

[34]  Ekaterini Ioannou,et al.  On Generating Benchmark Data for Entity Matching , 2012, Journal on Data Semantics.

[35]  Daniel J. Abadi,et al.  Scalable Semantic Web Data Management Using Vertical Partitioning , 2007, VLDB.

[36]  Axel-Cyrille Ngonga Ngomo,et al.  How Well Does Your Instance Matching System Perform? Experimental Evaluation with LANCE , 2016, BLINK@ISWC.

[37]  Sören Auer,et al.  LIMES - A Time-Efficient Approach for Large-Scale Link Discovery on the Web of Data , 2011, IJCAI.

[38]  Christian Bizer,et al.  The Berlin SPARQL Benchmark , 2009, Int. J. Semantic Web Inf. Syst..

[39]  Tom Heath,et al.  Linked Data: Evolving the Web into a Global Data Space , 2011, Linked Data.

[40]  Masaki Aono,et al.  Anchor-Flood: Results for OAEI 2009 , 2009, OM.

[41]  Jérôme David,et al.  The Alignment API 4.0 , 2011, Semantic Web.

[42]  Martin Gaedke,et al.  Discovering and Maintaining Links on the Web of Data , 2009, SEMWEB.

[43]  Dimitris Plexousakis,et al.  OtO Matching System: A Multi-strategy Approach to Instance Matching , 2012, CAiSE.

[44]  Juan-Zi Li,et al.  RiMOM2013 results for OAEI 2013 , 2013, OM.

[45]  Stefan Conrad,et al.  A Benchmark for Testing Instance-based Ontology Matching Methods , 2010, EKAW.

[46]  Valerie V. Cross,et al.  LogMap family results for OAEI 2014 , 2014, OM.

[47]  Jeff Heflin,et al.  LUBM: A benchmark for OWL knowledge base systems , 2005, J. Web Semant..

[48]  Abderrahmane Khiat,et al.  InsMT / InsMTL results for OAEI 2014 instance matching , 2014, OM.

[49]  Renée J. Miller,et al.  Linkage Query Writer , 2009, Proc. VLDB Endow..

[50]  Nathalie Pernelle,et al.  LN2R a knowledge based reference reconciliation system: OAEI 2010 results , 2010, OM.

[51]  Ioana Manolescu,et al.  Performance evaluation in database research: principles and experience , 2009, EDBT '09.

[52]  Octavian Udrea,et al.  Apples and oranges: a comparison of RDF benchmarks and real RDF datasets , 2011, SIGMOD '11.

[53]  Nicole Redaschi.  UniProt in RDF: Tackling Data Integration and Distributed Annotation with the Semantic Web , 2009 .

[54]  Ioana Manolescu,et al.  XMark: A Benchmark for XML Data Management , 2002, VLDB.

[55]  Yuzhong Qu,et al.  ObjectCoref & Falcon-AO: results for OAEI 2010 , 2010, OM.

[56]  Georg Lausen,et al.  SP^2Bench: A SPARQL Performance Benchmark , 2008, 2009 IEEE 25th International Conference on Data Engineering.