Using Segmented Objects in Ostensive Video Shot Retrieval

This paper presents a system for video shot retrieval in which shots are retrieved based on matching video objects using a combination of colour, shape and texture. Rather than matching on individual objects, our system supports sets of query objects which in total reflect the user's object-based information need. Our work also adapts to a shifting user information need by initiating the partitioning of a user's search into two or more distinct search threads, which can be followed by the user in sequence. This is an automatic process which maps neatly to the ostensive model for information retrieval in that it allows a user to place a virtual checkpoint on their search, explore one thread or aspect of their information need and then return to that checkpoint to then explore an alternative thread. Our system is fully functional and operational and in this paper we illustrate several design decisions we have made in building it.

[1]  Alan F. Smeaton,et al.  TRECVID 2004 Experiments in Dublin City University , 2004, TRECVID.

[2]  Alan F. Smeaton,et al.  TRECVid 2006 Experiments at Dublin City University , 2012, TRECVID.

[3]  Kristiina Jokinen,et al.  Explanation and Interaction: The computer generation of Explanatory Dialogues , 1996, Machine Translation.

[4]  Paul Over,et al.  TRECVID: evaluating the effectiveness of information retrieval tasks on digital video , 2004, MULTIMEDIA '04.

[5]  B. S. Manjunath,et al.  Texture Features for Browsing and Retrieval of Image Data , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  B. S. Manjunath,et al.  Introduction to MPEG-7: Multimedia Content Description Interface , 2002 .

[7]  Iain Campbell,et al.  The ostensive model of developing information needs , 2000 .

[8]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[9]  Nicholas J. Belkin,et al.  On the nature and fuction of explanation in intelligent information retrieval , 1988, SIGIR '88.

[10]  Stefan M. Rüger,et al.  Three Interfaces for Content-Based Access to Image Collections , 2004, CIVR.

[11]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[12]  Qiang Yang,et al.  A unified framework for semantics and feature based relevance feedback in image retrieval systems , 2000, ACM Multimedia.

[13]  Konstantinos N. Plataniotis,et al.  Query feedback for interactive image retrieval , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Mounia Lalmas,et al.  A survey on the use of relevance feedback for information access systems , 2003, The Knowledge Engineering Review.

[15]  Ian Ruthven.  On the Use of Explanations as Mediating Device for Relevance Feedback , 2002, ECDL.

[16]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Raya Fidel,et al.  Ranking expansion terms using partial and ostensive evidence , 2002 .

[18]  T. Moon.  The expectation-maximization algorithm , 1996, IEEE Signal Process. Mag..

[19]  Marcel Worring,et al.  Interactive Search Using Indexing, Filtering, Browsing and Ranking , 2003, TRECVID.

[20]  John R. Smith,et al.  MPEG-7 multimedia description schemes , 2001, IEEE Trans. Circuits Syst. Video Technol..