CRAWLER-LD: A Multilevel Metadata Focused Crawler Framework for Linked Data

The Linked Data best practices recommend to publish a new tripleset using well-known ontologies and to interlink the new tripleset with other triplesets. However, both are difficult tasks. This paper describes CRAWLER-LD, a metadata crawler that helps selecting ontologies and triplesets to be used, respectively, in the publication and the interlinking processes. The publisher of the new tripleset first selects a set T of terms that describe the application domain of interest. Then, he submits T to CRAWLER-LD, which searches for triplesets whose vocabularies include terms direct or transitively related to those in T. CRAWLER-LD returns a list of ontologies to be used for publishing the new tripleset, as well as a list of triplesets that the new tripleset can be interlinked with. CRAWLER-LD focuses on specific metadata properties, including subclass of, and returns only metadata, hence the classification “metadata focused crawler”.