A consensual dataset for complex ontology matching evaluation

Simple ontology alignments, largely studied in the literature, link one single entity of a source ontology to one single entity of a target ontology. One of the limitations of these alignments is, however, their lack of expressiveness, which can be overcome by complex alignments, which are composed of correspondences involving logical constructors or transformation functions. While most work on complex ontology matching has been dedicated to the development of complex matching approaches, there is still a lack of benchmarks on which the complex approaches can be systematically evaluated. The aim of this paper is to present the process of constructing the consensual complex Conference dataset, describing the design choices and the methodology followed for constructing it. We discuss the issues the experts were faced with during the process and discuss the lessons learned and perspectives in the field.

[1]  Johanna Völker,et al.  Linguistic analysis for complex ontology matching , 2010, OM.

[2]  Ollivier Haemmerlé,et al.  Survey on complex ontology matching , 2020, Semantic Web.

[3]  Susan Elliott Sim,et al.  Using benchmarking to advance research: a challenge to software engineering , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[4]  Pascal Hitzler,et al.  Conference v2.0: An Uncertain Version of the OAEI Conference Benchmark , 2014, SEMWEB.

[5]  Fausto Giunchiglia,et al.  Save Up to 99% of Your Time in Mapping Validation , 2010, OTM Conferences.

[6]  Pedro M. Domingos,et al.  iMAP: discovering complex semantic matches between database schemas , 2004, SIGMOD '04.

[7]  Gem Stapleton,et al.  A Vision for Diagrammatic Ontology Engineering , 2014, VISUAL@EKAW.

[8]  Ernesto Jiménez-Ruiz,et al.  Evaluating Ontology Alignment Systems in Query Answering Tasks , 2014, International Semantic Web Conference.

[9]  Catia Pesquita,et al.  Improving the interoperability of biomedical ontologies with compound alignments , 2018, Journal of Biomedical Semantics.

[10]  Catherine Roussey,et al.  Cross-Querying LOD Datasets Using Complex Alignments: An Application to Agronomic Taxa , 2017, MTSR.

[11]  Nektarios Gioldasis,et al.  SPARQL-RW: transparent query access over mapped RDF data sources , 2012, EDBT '12.

[12]  Willem Robert van Hage,et al.  Sample Evaluation of Ontology-Matching Systems , 2007, EON.

[13]  Heiner Stuckenschmidt,et al.  Results of the Ontology Alignment Evaluation Initiative 2007 , 2006, OM.

[14]  Heiner Stuckenschmidt,et al.  A Pattern-based Ontology Matching Approach for Detecting Complex Correspondences , 2009, OM.

[15]  Christian Meilicke,et al.  Alignment incoherence in ontology matching , 2011 .

[16]  Erhard Rahm,et al.  Comparison of Schema Matching Evaluations , 2002, Web, Web-Services, and Database Systems.

[17]  J. Euzenat,et al.  Ontology Matching , 2007, Springer Berlin Heidelberg.

[18]  Heiner Stuckenschmidt,et al.  Results of the Ontology Alignment Evaluation Initiative , 2007 .

[19]  Ollivier Haemmerlé,et al.  Complex matching based on competency questions for alignment: a first sketch , 2018, OM@ISWC.

[20]  Christophe Debruyne,et al.  Extension of the M-Gov ontology mapping framework for increased traceability , 2017, OM@ISWC.

[21]  Bernardo Pereira Nunes,et al.  Complex matching of RDF datatype properties , 2011, OM.

[22]  Deborah L. McGuinness,et al.  When owl: sameAs Isn't the Same: An Analysis of Identity in Linked Data , 2010, SEMWEB.

[23]  Ollivier Haemmerlé,et al.  Task-Oriented Complex Ontology Alignment: Two Alignment Evaluation Sets , 2018, ESWC.

[24]  V. Svátek,et al.  OntoFarm : Towards an Experimental Collection of Parallel Ontologies , 2005 .

[25]  Jane Huffman Hayes,et al.  Good Benchmarks are Hard To Find: Toward the Benchmark for Information Retrieval Applications in Software Engineering , 2006 .

[26]  Declan O'Sullivan,et al.  Bayes-ReCCE: A Bayesian Model for Detecting Restriction Class Correspondences in Linked Open Data Knowledge Bases , 2016, Int. J. Semantic Web Inf. Syst..

[27]  Boris Motik,et al.  MAFRA - A MApping FRAmework for Distributed Ontologies , 2002, EKAW.

[28]  Ondrej Sváb-Zamazal,et al.  The First Version of the OAEI Complex Alignment Benchmark , 2018, International Semantic Web Conference.

[29]  Pierre Zweigenbaum,et al.  MEANS : une approche sémantique pour la recherche de réponses aux questions médicales , 2014, Trait. Autom. des Langues.

[30]  Vojtech Svátek,et al.  The Ten-Year OntoFarm and its Fertilization within the Onto-Sphere , 2017, J. Web Semant..

[31]  Craig A. Knoblock,et al.  Discovering Concept Coverings in Ontologies of Linked Data Sources , 2012, International Semantic Web Conference.

[32]  Jiawei Han,et al.  Discovering complex matchings across web query interfaces: a correlation mining approach , 2004, KDD.

[33]  Pascal Hitzler,et al.  A Complex Alignment Benchmark: GeoLink Dataset , 2018, International Semantic Web Conference.

[34]  Bob J. Wielinga,et al.  Let's agree to disagree: on the evaluation of vocabulary alignment , 2011, K-CAP '11.

[35]  Dejing Dou,et al.  Ontology Matching with Knowledge Rules , 2015, Trans. Large Scale Data Knowl. Centered Syst..

[36]  Dejing Dou,et al.  Discovering Executable Semantic Mappings Between Ontologies , 2007, OTM Conferences.

[37]  Rodrigo Bonacin,et al.  Ontology Mapping Validation: Dealing with an NP-Complete Problem , 2011, Int. J. Web Portals.

[38]  Giovanna Guerrini,et al.  Minimizing conservativity violations in ontology alignments: algorithms and evaluation , 2016, Knowledge and Information Systems.

[39]  Isabel F. Cruz,et al.  Results of AML participation in OAEI 2018 , 2018, OM@ISWC.

[40]  Carsten Binnig,et al.  RODI: Benchmarking relational-to-ontology mapping generation quality , 2017, Semantic Web.

[41]  Craig A. Knoblock,et al.  Linking and Building Ontologies of Linked Data , 2010, SEMWEB.

[42]  Trevor J. M. Bench-Capon,et al.  An Analysis of Ontology Mismatches; Heterogeneity versus Interoperability , 2007 .

[43]  Antoine Isaac,et al.  Two Variations on Ontology Alignment Evaluation: Methodological Issues , 2008, ESWC.

[44]  Heiner Stuckenschmidt,et al.  A Reasoning-Based Support Tool for Ontology Mapping Evaluation , 2009, ESWC.

[45]  Robert Stevens,et al.  Measuring Expert Performance at Manually Classifying Domain Entities under Upper Ontology Classes , 2018, J. Web Semant..

[46]  Claus Zinn,et al.  Putting Ontology Alignment in Context: Usage Scenarios, Deployment and Evaluation in a Library Case , 2008, ESWC.

[47]  Declan O'Sullivan,et al.  Milan: Automatic Generation of R2RML Mappings , 2018, AICS.

[48]  Heiner Stuckenschmidt,et al.  Incoherence as a Basis for Measuring the Quality of Ontology Mappings , 2008, OM.