Diogenes: A Web Search Agent for Content Based Indexing of Personal Images

Content based indexing of multimedia has always been a challenging task. Especially in webbased retrieval systems, a key component in meeting this challenge is to be able to deal with imprecise and uncertain information e ectively. In this work, we examine evidence combination mechanisms for classifying multimedia information retrieved from the web. In particular, we examine linear and Dempster-Shafer methods of evidence combination in the context of identifying personal images on the World Wide Web. An automatic web search agent named Diogenes searches the web for personal images and combines di erent pieces of evidence for identi cation. The sources of evidence consist of input from face recognition and text/HTML analysis modules. A degree of uncertainty is involved with both of these sources. Diogenes automatically determines the uncertainty locally for each retrieval and uses this information to set a relative signi cance for each evidence. To our knowledge, Diogenes is the rst image search agent using Dempster-Shafer evidence combination based on automatic object recognition and dynamic local uncertainty assessment. In this paper we report the results of two sets of experiments where Diogenes was compared against some well known web image search engines. Diogenes has comfortably outperformed these search engines for celebrity image queries in terms of average precision. We also evaluate under which conditions the evidence combination methods described yielded optimum bene t.