Introduction to the special issue on database and information retrieval integration

The goal of having a common platform for dealing with both structured and unstructured data is a longstanding one, going back to the 1960s. A number of approaches have been suggested, both from the database and information retrieval (IR) perspective, but the motivation for finding a solution or solutions that work has grown tremendously since the advent of very large-scale Web databases. Areas that were once the exclusive concerns of IR such as statistical inference and ranking, have now become important topics for database researchers and both communities have a common interest in providing efficient indexing and optimization techniques for Web-scale data. Exploiting document structure is a critical part of Web search and combining different sources of evidence effectively is an important part of many database applications. There are many possibilities for integration such as extending a database model to more effectively deal with probabilities, extending an IR model to handle more complex structures and multiple relations, or developing a unified model and system. Applications such as Web search, e-commerce, and data mining, provide the testbeds where these proposals can be evaluated and compared. The papers in this special issue cover a range of topics related to database and IR integration. To provide some context, it is worth briefly reviewing some of the work that was done in the past, particularly in the more distant pre-Web days. From an IR perspective, dealing with structure started in the 1970s with commercial search services such as MEDLINE and DIALOG that had Boolean field restrictions.