Generating semantic visual templates for video databases

We describe a system that generates semantic visual templates (SVTs) for video databases. From a single query sketch, new queries are automatically generated with each one representing a different view of the initial sketch. The combination of the original and new queries forms a large set of potential queries for a content-based video retrieval system. Through Bayesian relevance feedback, the user narrows the choices to an exemplar set. This exemplar set, or SVTs, represents personalized views of a concept and an effective set of queries to retrieve a general category of images and videos. We have generated SVTs for several classes of videos, including sunsets, high jumpers, and slalom skiers. Our experiments show that the user can quickly converge upon SVTs with optimal performance, achieving over 85% of the precision from icons chosen by exhaustive search.