Search engines on the Web are valuable tools for searching information according to a user's interests whether an individual or a software agent. In the present article we describe the design and the operation mode of SpiderServer, a metasearch engine used for the submission of a query followed by the retrieving of results from five popular search engines. SpiderServer is the metasearch engine of the WebNaut system but it can be easily used by any other metasearch platform. There are two files for every search engine describing the phases of query formation and filtering respectively. These files contain directions on the way a query must be modified for a specific search engine and on the methodology SpiderServer must follow in order to parse the results from the specific search engine. The ultimate goal is to construct platform independent meta-search engines, which can be easily programmed to adapt to any search engine available on the WEB.
[1]
Oren Etzioni,et al.
The MetaCrawler architecture for resource aggregation on the Web
,
1997
.
[2]
Gerhard X. Ritter,et al.
An Overview of World Wide Web Search Technologies
,
2001
.
[3]
Adele E. Howe,et al.
SAVVYSEARCH: A Metasearch Engine That Learns Which Search Engines to Query
,
1997,
AI Mag..
[4]
Z. Z. Nick,et al.
Web search using a genetic algorithm
,
2001
.
[5]
Themis Panayiotopoulos,et al.
Web Search Using a Genetic Algorithm
,
2001,
IEEE Internet Comput..
[6]
Themis Panayiotopoulos,et al.
A metagenetic algorithm for information filtering and collection from the World Wide Web
,
2001,
Expert Syst. J. Knowl. Eng..
[7]
Adele E. Howe,et al.
Experiences with selecting search engines using metasearch
,
1997,
TOIS.
[8]
Luis Gravano,et al.
STARTS: Stanford Protocol Proposal for Internet Retrieval and Search
,
1997
.