How Well Does Your Instance Matching System Perform? Experimental Evaluation with LANCE

Identifying duplicate instances in the Data Web is most commonly performed (semi-)automatically using instance matching frameworks. However, current instance matching benchmarks fail to provide end users and developers with the necessary insights pertaining to how current frameworks behave when dealing with real data. In this paper, we present the results of the evaluation of instance matching systems using Lance, a domain-independent, schema agnostic instance matching benchmark generator for Linked Data. Lance is the first benchmark generator for Linked Data to support semantics-aware test cases that take into account complex OWL constructs in addition to the standard test cases related to structure and value transformations. We provide a comparative analysis with benchmarks produced using the Lance framework for different domains to assess and identify the capabilities of state of the art instance matching systems.

[1]  Erhard Rahm,et al.  Evolution of the COMA match system , 2011, OM.

[2]  Yi Li,et al.  RiMOM: A Dynamic Multistrategy Ontology Alignment Framework , 2009, IEEE Transactions on Knowledge and Data Engineering.

[3]  Wang Chiew Tan,et al.  STBenchmark: towards a benchmark for mapping systems , 2008, Proc. VLDB Endow..

[4]  Bernardo Cuenca Grau,et al.  LogMap: Logic-Based and Scalable Ontology Matching , 2011, SEMWEB.

[5]  Li Ma,et al.  Towards a Complete OWL Ontology Benchmark , 2006, ESWC.

[6]  Ian Horrocks,et al.  MORe: a Modular OWL Reasoner for Ontology Classification , 2013, ORE.

[7]  Sören Auer,et al.  LIMES - A Time-Efficient Approach for Large-Scale Link Discovery on the Web of Data , 2011, IJCAI.

[8]  Axel-Cyrille Ngonga Ngomo,et al.  LANCE: Piercing to the Heart of Instance Matching Tools , 2015, SEMWEB.

[9]  Vasilis Efthymiou,et al.  Entity resolution in the web of data , 2013, Entity Resolution in the Web of Data.

[10]  Stefan Conrad,et al.  A Benchmark for Testing Instance-based Ontology Matching Methods , 2010, EKAW.

[11]  Dimitris Plexousakis,et al.  OtO Matching System: A Multi-strategy Approach to Instance Matching , 2012, CAiSE.

[12]  Heiner Stuckenschmidt,et al.  Results of the Ontology Alignment Evaluation Initiative 2007 , 2006, OM.

[13]  Robert Isele,et al.  Silk Server - Adding missing Links while consuming Linked Data , 2010, COLD.

[14]  Heiner Stuckenschmidt,et al.  Results of the Ontology Alignment Evaluation Initiative , 2007 .

[15]  Axel-Cyrille Ngonga Ngomo,et al.  Pushing the Limits of Instance Matching Systems: A Semantics-Aware Benchmark for Linked Data , 2015, WWW.

[16]  Axel-Cyrille Ngonga Ngomo,et al.  EAGLE: Efficient Active Learning of Link Specifications Using Genetic Programming , 2012, ESWC.