Evaluation of SPARQL-compliant semantic search user interfaces

A regular user of a semantic search system frequently posses no knowledge about the SPARQL language nor about the ontology of a given knowledge base, especially when it provides domain-unspecific data obtained from heterogeneous sources. Nevertheless, he/she should be provided with tools enabling both intuitive and effective exploration of RDF-compliant knowledge bases. Natural language querying is one of the solutions that have been proposed so far as means for making knowledge bases more user-friendly. However, the results of natural language querying usually have lower precision and recall than analogical results of graph-based querying. In the paper, we introduce an evaluation methodology based on the 2011 QALD workshop queries that allows to measure the accuracy of a semantic search system as well as the complexity of the query formulation process. The obtained results confirm the intuition that graph-based querying, although assuring comparatively high accuracy of the results, is usually still too difficult for regular users. On the other hand, on the basis of results obtained for an experimental search system referred to as Semantic Focused Crawler, we claim that enhancing a SPARQL-compliant graph-based system by an entity-type recommendation feature may reduce the number of query elements necessary to formulate a query without compromising the quality of the results.