Extending relevance feedback with a group-based user interface

In the traditional user interfaces of Query-by-Example (QbE) in Content-Based Image Retrieval (CBIR), finding good combinations of query examples is essential for successful retrieval. Unfortunately, the traditional user interfaces are not suitable for trying different combinations of query examples. These systems assumed query examples are added incrementally. The only way to refine the query was to add more example images. When no additional example is found in the result set, the search is considered to have converged. Furthermore, no place was provided to hold previous query results. We are developing ImageGrouper, a new interface for Content-based image retrieval. In this system, the users can interactively compare different combinations of query examples by dragging and grouping images on the workspace (Query-by-Group.) Unlike the traditional systems, a group of images is considered as the basic unit of the query. Because the query results are displayed on another pane, the user can quickly review the results. Combining different query results is also easier. ImageGrouper makes possible new image search methods that were difficult in the traditional user interfaces. First, since the user can annotate text information on each group, integration of keyword-based and content-based search becomes easy. Second, by creating a hierarchy of query examples, the user can begin with collecting relatively generic image first, then narrow down the search to more specific images. Finally, the user can create multiple groups of positive examples. Therefore, we can extend relevance feedback algorithms from two-classes problem (positive and negative) to multiple-classes problem (multiple positive and negative classes.)