Web-scale semantic information processing

From the earliest days of semantic web research, the problem of semantically processing information at very large scale has cast a shadow over its many successes. Having the Web as a primary use case has been both a blessing and a curse. While targeting the Web has attracted a lot of attention and has significantly improved the awareness for the benefits of semantic data models and automatic reasoning, the field still has to prove that semantics will work on web scale. During the first years, semantic web research was dominated by on the consideration of rich semantic representations in the tradition of symbolic AI. Significant progress was achieved with respect to integrating knowledge representation and reasoning with mainstream web infrastructure leading to key standards such as OWL, RDF and SPARQL, however, processing enormous quantities of the corresponding data is still one of the greatest challenges for the Semantic Web. While other communities, e.g. Information Retrieval, have developed successful strategies for coping with the scale of the Web using statistical techniques, semantic web technologies are still struggling with scaling up to the Web as such. This is in part due to the need to preserve the data’s structure and the need to perform various forms of reasoning in order to more effectively leverage the available information. In order to cope with these challenges it is necessary to look beyond the realms of artificial intelligence and to leverage ideas and techniques from the distributed systems and the database community.