GeniMiner: Web Mining with a Genetic-Based Algorithm

We present in this paper a genetic search strategy for a search engine. We begin by showing that important relations exist between Web statistical studies, search engines, and standard techniques in optimization: the web is a graph which can be searched for relevant information with an evaluation function and with operators based on standard search engines or local exploration. It is then straightforward to define an evaluation function that is a mathematical formulation of the user request and to define a steady state genetic algorithm that evolves a population of pages with binary tournament selection and specific operators. The creation of individuals is performed by querying standard search engines. The mutation operator consists in exploring the neighborhood of a page thanks to the links going out of that page. We present a comparative evaluation which is performed with the same protocol as used in optimization. Our tool obtains pages which are significantly better than those found by standard search engines for complex queries. We conclude by showing that our framework for Web search could be generalized to other optimization t echniques like parallel genetic algorithms.