Group-based interface for content-based image retrieval

In Content-based Image Retrieval (CBIR) systems, the Query-by-Example (QBE) approach is commonly used. However, because of inevitable "semantic gaps" between visual features and the user's concepts, trial-and-error query is essential for successful retrieval. Unfortunately, traditional user interfaces are not suitable for trying different combinations of query examples. This is because in these systems, query specification and result display are done on the same workspace. Once the user removes an image from the query examples, the image may disappear from the user interface. In addition, it is difficult to combine the result of different queries. In this paper, we propose a new interface for Content-based image retrieval. In our system, the users can interactively compare different combinations of query examples by dragging and grouping images on the workspace (Query-by-Group.) Because the query results are displayed on another pane, the user can quickly review the results. Combining different queries is also easy. Furthermore, the concept of "image groups" is also applied to annotating and organizing a large number of images. Because the gestural operations of our system is similar to file operations of modern window-based operation systems, users can easily learn to use the system.

[1]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[2]  Shih-Fu Chang,et al.  VisualSEEk: a fully automated content-based image query system , 1997, MULTIMEDIA '96.

[3]  Thomas S. Huang,et al.  Optimizing learning in image retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[4]  John C. Dalton,et al.  Hierarchical browsing and search of large image databases , 2000, IEEE Trans. Image Process..

[5]  Steve Jones.  Graphical query specification and dynamic result previews for a digital library , 1998, UIST '98.

[6]  Marcia J. Bates,et al.  The design of browsing and berrypicking techniques for the online search interface , 1989 .

[7]  Ben Shneiderman,et al.  Readings in information visualization - using vision to think , 1999 .

[8]  Kerry Rodden,et al.  Does organisation by similarity assist image browsing? , 2001, CHI.

[9]  Gregory J. Wolff,et al.  Storytelling with digital photographs , 2000, CHI.

[10]  Andreas Paepcke,et al.  The digital library integrated task environment (DLITE) , 1997, DL '97.

[11]  Henning Müller,et al.  Automated benchmarking in content-based image retrieval , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[12]  Christos Faloutsos,et al.  MindReader: Querying Databases Through Multiple Examples , 1998, VLDB.

[13]  Michael L. Creech,et al.  FotoFile: a consumer multimedia organization and retrieval system , 1999, CHI '99.

[14]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[15]  Robin Jeffries,et al.  Orienteering in an information landscape: how information seekers get from here to there , 1993, INTERCHI.

[16]  Thomas S. Huang,et al.  3D MARS: immersive virtual reality for content-based image retrieval , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[17]  Erkki Oja,et al.  Content-Based Image Retrieval Using Self-Organizing Maps , 1999, VISUAL.

[18]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Thomas S. Huang,et al.  Comparing discriminating transformations and SVM for learning during multimedia retrieval , 2001, MULTIMEDIA '01.

[20]  Simone Santini,et al.  Integrated browsing and querying for image databases , 2000, IEEE MultiMedia.

[21]  Thomas S. Huang,et al.  Edge-based structural features for content-based image retrieval , 2001, Pattern Recognit. Lett..

[22]  Benjamin B. Bederson,et al.  Quantum Treemaps and Bubblemaps for a Zoomable Image Browser , 2001 .

[23]  Shih-Fu Chang,et al.  Transform features for texture classification and discrimination in large image databases , 1994, Proceedings of 1st International Conference on Image Processing.

[24]  Minh N. Do,et al.  Integrated Browsing and Searching of Large Image Collections , 2000, VISUAL.

[25]  Ben Shneiderman,et al.  Direct annotation: a drag-and-drop strategy for labeling photos , 2000, 2000 IEEE Conference on Information Visualization. An International Conference on Computer Visualization and Graphics.

[26]  Ingemar J. Cox,et al.  The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments , 2000, IEEE Trans. Image Process..

[27]  Markus A. Stricker,et al.  Similarity of color images , 1995, Electronic Imaging.

[28]  Paul Over,et al.  Comparing interactive information retrieval systems across sites: the TREC-6 interactive track matrix experiment , 1998, SIGIR '98.