Video retrieval using objects and ostensive relevance feedback

The thesis discusses and evaluates a model of video information retrieval that incorporates a variation of Relevance Feedback and facilitates object-based interaction and ranking. Video and image retrieval systems suffer from poor retrieval performance compared to text-based information retrieval systems and this is mainly due to the poor discrimination power of visual features that provide the search index. Relevance Feedback is an iterative approach where the user provides the system with relevant and non-relevant judgements of the results and the system re-ranks the results based on the user judgements. Relevance feedback for video retrieval can help overcome the poor discrimination power of the features with the user essentially pointing the system in the right direction based on their judgements. The ostensive relevance feedback approach discussed in this work weights user judgements based on the o r d e r in which they are made with newer judgements weighted higher than older judgements. The main aim of the thesis is to explore the benefit of ostensive relevance feedback for video retrieval with a secondary aim of exploring the effectiveness of object retrieval. A user experiment has been developed in which three video retrieval system variants are evaluated on a corpus of video content. The first system applies standard relevance feedback weighting while the second and third apply ostensive relevance feedback with variations in the decay weight. In order to evaluate effective object retrieval, animated video content provides the corpus content for the evaluation experiment as animated content offers the highest performance for object detection and extraction.

[1]  S. Marlow,et al.  A combined audio-visual contribution to event detection in field sports broadcast video. Case study: Gaelic football , 2003, Proceedings of the 3rd IEEE International Symposium on Signal Processing and Information Technology (IEEE Cat. No.03EX795).

[2]  Donna K. Harman,et al.  The DARPA TIPSTER project , 1992, SIGF.

[3]  J. J. Rocchio,et al.  Relevance feedback in information retrieval , 1971 .

[4]  Jakob Nielsen,et al.  Improving a human-computer dialogue , 1990, CACM.

[5]  Noel E. O'Connor,et al.  User interface design for keyframe-based browsing of digital video , 2001 .

[6]  Donna Harman,et al.  Overview of the First Text REtrieval Conference. , 1993, SIGIR 1993.

[7]  Neel Sundaresan,et al.  Web-based searching and browsing of multimedia data , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[8]  Cyril W. Cleverdon,et al.  Factors determining the performance of indexing systems , 1966 .

[9]  Adrian David Cheok,et al.  22nd International Conference on Human-Computer Interaction with Mobile Devices and Services , 2007, Lecture Notes in Computer Science.

[10]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Noel E. O'Connor,et al.  Evaluating and combining digital video shot boundary detection algorithms , 2000 .

[12]  Milind R. Naphade,et al.  Video retrieval and relevance feedback in the context of a post-integration model , 2001, 2001 IEEE Fourth Workshop on Multimedia Signal Processing (Cat. No.01TH8564).

[13]  Marcia J. Bates,et al.  The design of browsing and berrypicking techniques for the online search interface , 1989 .

[14]  N. O'Connor,et al.  Rhythm detection for speech-music discrimination in MPEG compressed domain , 2002, 2002 14th International Conference on Digital Signal Processing Proceedings. DSP 2002 (Cat. No.02TH8628).

[15]  Alan F. Smeaton,et al.  The Físchlár-News-Stories System: Personalised Access to an Archive of TV News , 2004, RIAO.

[16]  Shih-Fu Chang,et al.  Querying by color regions using VisualSEEk content-based visual query system , 1997 .

[17]  HongJiang Zhang,et al.  MSR-Asia at TREC-10 Video Track: Shot Boundary Detection Task , 2001, TREC.