Automatic Performance Evaluation of Web Search Systems using Rough Set based Rank Aggregation

Web searching is such an activity that its importance can just not be ignored in the current scenario. Since there are a large number of publicly accessible search engines, shich differ in their indexing algorithms and hence the search results, the evaluation of search engines performance is needed to determine which one is the best. The human intelligence may be used to measure the search engine effectiveness. But, a subjective evaluation done on the basis of user-feedback is costly in terms of the time required. Therefore, it is also not scalable. So, there is a need of an automatic evaluation method. In this paper, we present the architecture of an automatic Web search evaluation system that combines the different evaluation techniques using a Rough Set based Rank aggregation technique. The rough set based rank aggregation models the user’s feedback based rank aggregation. In the rough set based aggregation technique, the ranking rules are learnt on the basis of the user feedback in the training data sets. The learned rules are then used to estimate the overall ranking for the other data sets, for which user feedback is not available. We show our experimental results pertaining to seven public search engines.