A Methodology for Retrieving Text Documents from Multiple Databases. (submitted for Publication.) Automatic Retrieval with Locality Information Using 6.3.1 Document Fetching 6.2 Similarity Adjustment 5.4 Learning-based Approaches 5.1 Local Determination 5.2 User Determination 5 Select Documents from

report discussed how to estimate the database usefulness deened in this paper for the high-correlation and disjoint scenarios. Such discussions did not appear in 18].) a collection of up to more than 6,000 real Internet queries is used. However, the database collection is small and there is no document relevance information. An ideal testbed should have a large collection of databases of various sizes, contents and structures, a large collection of queries of various lengths and with the relevant documents for each query identiied. A recently proposed testbed for evaluating metasearch techniques 14] is far from being an ideal testbed. The above list of challenges is by no means complete. New problems will arise with more understanding of the issues in metasearch. 8 Conclusions With the increase of the number of search engines and digital libraries on the World Wide Web, providing easy, eecient and eeective access to text information from multiple sources has increasingly become necessary. In this article, we presented an overview of existing metasearch techniques. Our overview concentrated on the problems of database selection, document selection and result merging. A wide variety of techniques for each of these problems were surveyed and compared. We also discussed the underlying causes for these problems to be so challenging: various hetero-geneities among diierent local search engines due to the independent implementations of these search engines, and the lack of information about these implementations because they are mostly proprietary. Our survey and investigation seem to indicate that there may not be a single best solution for any of the main problems focused in this article, namely, the database selection problem, the document selection problem and the result merging problem. Better solutions often require more information from local search engines such as more detailed database representatives, underlying similarity functions, term weighting schemes, indexing methods, and so on. There are currently no suuciently eecient methods to nd such information independently. A possible scenario is that we will need good solutions based on diierent degrees of knowledge about each local search engine and then apply these solutions accordingly. Another important issue is the scalability of the solutions. Ultimately, we need to develop solutions that can scale in two orthogonal dimensions: data and access. Speciically, a good solution must scale to thousands of databases with many of them containing millions of documents and to millions of accesses a day. None of the proposed solutions have …