Developing a Benchmark Suite for Semantic Web Data from Existing Workflows

This paper presents work in progress towards developing a new benchmark for federated query processing systems. Unlike other popular benchmarks, our queryset is not driven by technical evaluation, but is derived from workflows established by the pharmacology community. The value of this queryset is that it is realistic but at the same time it comprises complex queries that test all features of modern query processing systems.

[1]  Katja Hose,et al.  FedX: A Federation Layer for Distributed Query Processing on Linked Open Data , 2011, ESWC.

[2]  Günter Ladwig,et al.  FedBench: A Benchmark Suite for Federated Semantic Data Query Processing , 2011, SEMWEB.

[3]  Antonis Troumpoukis,et al.  SemaGrow: optimizing federated SPARQL queries , 2015, SEMANTiCS.

[4]  Paul T. Groth,et al.  On the formulation of performant SPARQL queries , 2013, J. Web Semant..

[5]  Jeff Heflin,et al.  LUBM: A benchmark for OWL knowledge base systems , 2005, J. Web Semant..

[6]  Steffen Staab,et al.  SPLENDID: SPARQL Endpoint Federation Exploiting VOID Descriptions , 2011, COLD.

[7]  Axel-Cyrille Ngonga Ngomo,et al.  LargeRDFBench: A billion triples benchmark for SPARQL endpoint federation , 2018, J. Web Semant..

[8]  Carole A. Goble,et al.  API-centric Linked Data integration: The Open PHACTS Discovery Platform case study , 2014, J. Web Semant..

[9]  Muhammad Saleem,et al.  UPSP: Unique Predicate-based Source Selection for SPARQL Endpoint Federation , 2016, PROFILES@ESWC.

[10]  Paul T. Groth,et al.  Drug discovery FAQs: workflows for answering multidomain drug discovery questions. , 2015, Drug discovery today.

[11]  Barbara Zdrazil,et al.  Scientific competency questions as the basis for semantically enriched open pharmacological space development. , 2013, Drug discovery today.

[12]  Dirk Merkel,et al.  Docker: lightweight Linux containers for consistent development and deployment , 2014 .

[13]  N. Shadbolt,et al.  4store: The Design and Implementation of a Clustered RDF Store , 2009 .