Experiments in interactive video search by addition and subtraction

We have developed an interactive video search system that allows the searcher to rapidly assess query results and easily pivot off those results to form new queries. The system is intended to maximize the use of the discriminative power of the human searcher. This is accomplished by providing a hierarchical segmentation, streamlined interface, and redundant visual cues throughout. The typical search scenario includes a single searcher with the ability to search with text and content-based queries. In this paper, we evaluate new variations on our basic search system. In particular we test the system using only visual content-based search capabilities, and using paired searchers in a realtime collaboration. We present analysis and conclusions from these experiments.

[1]  Yoshua Bengio,et al.  Convergence Properties of the K-Means Algorithms , 1994, NIPS.

[2]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Paul Over,et al.  Evaluation campaigns and TRECVid , 2006, MIR '06.

[4]  Jiebo Luo,et al.  Large-scale multimodal semantic concept detection for consumer video , 2007, MIR '07.

[5]  Rong Yan,et al.  Merging storyboard strategies and automatic retrieval for improving interactive video search , 2007, CIVR '07.

[6]  John Adcock,et al.  FXPAL at TRECVID 2006 , 2006, TRECVID.

[7]  Alan F. Smeaton,et al.  Físchlár-DiamondTouch: collaborative video searching on a table , 2006, Electronic Imaging.

[8]  Christian Petersohn.  Fraunhofer HHI at TRECVID 2004: Shot Boundary Detection System , 2004, TRECVID.

[9]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[10]  Dong Wang,et al.  Video search in concept subspace: a text-like paradigm , 2007, CIVR '07.

[11]  John Adcock,et al.  Interactive Video Search Using Multilevel Indexing , 2005, CIVR.

[12]  Marcel Worring,et al.  The challenge problem for automated detection of 101 semantic concepts in multimedia , 2006, MM '06.

[13]  Franciska de Jong,et al.  Annotation of Heterogeneous Multimedia Content Using Automatic Speech Recognition , 2007, SAMT.

[14]  Robert Chen,et al.  FXPAL at TRECVID 2005 , 2005, TRECVID.

[15]  John Adcock,et al.  FXPAL Experiments for TRECVID 2004 , 2004, TRECVID.

[16]  Javed A. Aslam,et al.  Models for metasearch , 2001, SIGIR '01.

[17]  Alan F. Smeaton,et al.  A usage study of retrieval modalities for video shot retrieval , 2006, Inf. Process. Manag..

[18]  Dennis Koelma,et al.  The MediaMill TRECVID 2008 Semantic Video Search Engine , 2008, TRECVID.

[19]  Susan T. Dumais,et al.  Using Linear Algebra for Intelligent Information Retrieval , 1995, SIAM Rev..

[20]  P. Good.  Permutation, Parametric, and Bootstrap Tests of Hypotheses (Springer Series in Statistics) , 1994 .

[21]  Rong Yan,et al.  Extreme video retrieval: joint maximization of human and computer performance , 2006, MM '06.

[22]  Marcel Worring,et al.  A Learned Lexicon-Driven Paradigm for Interactive Video Retrieval , 2007, IEEE Transactions on Multimedia.

[23]  Paul Over,et al.  TRECVID 2007--Overview , 2007, TRECVID.

[24]  Clifton Forlines,et al.  Rapid serial visual presentation techniques for consumer digital video devices , 2003, UIST '03.

[25]  Sheng Tang,et al.  Interactive Spatio-Temporal Visual Map Model for Web Video Retrieval , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[26]  Xuelong Li,et al.  Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.