CuZero: embracing the frontier of interactive visual search for informed users

Users of most visual search systems suffer from two primary sources of frustration. Before a search over this data is executed, a query must be formulated. Traditional keyword search systems offer only passive, non-interactive input, which frustrates users that are unfamiliar with the search topic or the target data set. Additionally, after query formulation, result inspection is often relegated to a tiresome, linear inspection of results bound to a single query. In this paper, we reexamine the struggles that users encounter with existing paradigms and present a solution prototype system, CuZero. CuZero employs a unique query process that allows zero-latency query formulation for an informed human search. Relevant visual concepts discovered from various strategies (lexical mapping, statistical occurrence, and search result mining) are automatically recommended in real time after users enter each single word. CuZero also introduces a new intuitive visualization system that allows users to navigate seamlessly in the concept space at-will and simultaneously while displaying the results corresponding to arbitrary permutations of multiple concepts in real time. The result is the creation of an environment that allows the user to rapidly scan many different query permutations without additional query reformulation. Such a navigation system also allows efficient exploration of different types of queries, such as semantic concepts, visual descriptors, and example content, all within one navigation session as opposed to the repetitive trials used in conventional systems.

[1]  Paul Over,et al.  Evaluation campaigns and TRECVid , 2006, MIR '06.

[2]  Shih-Fu Chang,et al.  Visual islands: intuitive browsing of visual search results , 2008, CIVR '08.

[3]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[4]  Marcel Worring,et al.  The Mediamill Semantic Video Search Engine , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[5]  Shih-Fu Chang,et al.  Columbia University's semantic video search engine , 2007, CIVR '07.

[6]  Michael G. Christel Examining user interactions with video retrieval systems , 2007, Electronic Imaging.

[7]  Shih-Fu Chang,et al.  A reranking approach for context-based concept fusion in video indexing and retrieval , 2007, CIVR '07.

[8]  Shih-Fu Chang,et al.  Revision of LSCOM Event/Activity Annotations , 2006 .

[9]  Shih-Fu Chang,et al.  Columbia University’s Baseline Detectors for 374 LSCOM Semantic Visual Concepts , 2007 .

[10]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[11]  Rong Yan,et al.  Learning query-class dependent weights in automatic video retrieval , 2004, MULTIMEDIA '04.

[12]  Tat-Seng Chua,et al.  VisionGo: bridging users and multimedia video retrieval , 2008, CIVR '08.

[13]  Shih-Fu Chang,et al.  Automatic discovery of query-class-dependent models for multimodal search , 2005, MULTIMEDIA '05.

[14]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[15]  John R. Smith,et al.  Multimedia semantic indexing using model vectors , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[16]  Rong Yan,et al.  Semantic concept-based query expansion and re-ranking for multimedia retrieval , 2007, ACM Multimedia.

[17]  Michael G. Christel,et al.  Exploiting multiple modalities for interactive video retrieval , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[18]  Edward Y. Chang,et al.  Support vector machine active learning for image retrieval , 2001, MULTIMEDIA '01.

[19]  Marcel Worring,et al.  Optimization of interactive visual-similarity-based search , 2008, TOMCCAP.

[20]  Tat-Seng Chua,et al.  TRECVID 2005 by NUS PRIS , 2005, TRECVID.

[21]  Behzad Shahraray,et al.  Searching Visual Semantic Spaces with Concept Filters , 2007 .

[22]  Dominic Abrams,et al.  Language, Speech, and Communication , 2006 .