Mining relevant information on the Web: a clique-based approach

The role of information management and retrieval in production processes has been gaining in importance in recent years. In this context, the ability to search for and quickly find the small piece of information needed from the huge amount of information available has crucial importance. One category of tools devoted to such a task is represented by search engines. Satisfying the basic needs of the Web user has led to the research of new tools that aim at helping more sophisticated users (communities, companies, interest groups) with more elaborate methods. An example is the use of clustering and classification algorithms or other specific data mining techniques. In such a context, the proper use of a thematic search engine is a crucial tool in supporting and orienting many activities. Several practical and theoretical problems arise in developing such tools, and we try to face some of these in this paper, extending previous work on Web mining. Here we consider two related problems: how to select an appropriate set of keywords for a thematic engine taking into account the semantic and linguistic extensions of the search context, and how to select and rank a subset of relevant pages given a set of search keywords. Both problems are solved using the same framework, based on a graph representation of the available information and on the search of particular node subsets of such a graph. Such subsets are effectively identified by a maximum-weight clique algorithm customized ad hoc for specific problems. The methods have been developed in the framework of a funded research project for the development of new Web search tools, they have been tested on real data, and are currently being implemented in a prototypal thematic search engine. The Web mining method presented in this paper can be applied to Web-based design and manufacturing.