A Novel Video Annotation Framework Based on Video Object

Video annotation is very important for video management, such as video retrieval. Despite continuous efforts in inventing new annotation algorithms, the annotation performance is usually unsatisfactory, and the annotation vocabulary is still limited due to the use of a small scale training set. In this paper, a novel video annotation framework based on the video object is presented, named Object-Based Video Annotation. By dividing video into three types, we deal with different kind of video in different way. The first kind of video was annotated by human base on the e-Annotation architecture. The second kind of video was automatically annotated by the web mining methods. The third kind of video annotated by video analysis model which detect the video object and label them at the same time. Then active learning model implement active learning method in the video database, which can add new labels and video in the database. We also present an application system based on annotations: video retrieval. At the same time we add relevance feedback in our framework to optimize the result. The system designed base on a real-world situation by including video gathered from the Internet and is designed for exploratory video retrieval system based on the internet.

[1]  Milind R. Naphade Statistical techniques in video data management , 2002, 2002 IEEE Workshop on Multimedia Signal Processing..

[2]  Thomas S. Huang,et al.  Factor graph framework for semantic video indexing , 2002, IEEE Trans. Circuits Syst. Video Technol..

[3]  Yi Wu,et al.  Ontology-based multi-classification learning for video concept detection , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[4]  John R. Smith,et al.  Multimedia semantic indexing using model vectors , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[5]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[6]  Alan F. Smeaton,et al.  User evaluation of Físchlár-News , 2006, ACM Trans. Inf. Syst..

[7]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[8]  Alan F. Smeaton Techniques used and open challenges to the analysis, indexing and retrieval of digital video , 2007, Inf. Syst..

[9]  Alan F. Smeaton,et al.  Large Scale Evaluations of Multimedia Information Retrieval: The TRECVid Experience , 2005, CIVR.

[10]  Marcel Worring,et al.  The challenge problem for automated detection of 101 semantic concepts in multimedia , 2006, MM '06.

[11]  Timo Ojala,et al.  Cluster-temporal browsing of large news video databases , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[12]  Ravin Balakrishnan,et al.  Fluid interaction techniques for the control and annotation of digital video , 2003, UIST '03.

[13]  G. Griffin,et al.  Caltech-256 Object Category Dataset , 2007 .

[14]  Xian-Sheng Hua,et al.  Online multi-label active annotation: towards large-scale content-based video search , 2008, ACM Multimedia.

[15]  John P. Eakins,et al.  Towards intelligent image retrieval , 2002, Pattern Recognit..

[16]  Amit Kumar Das,et al.  Image retrieval based on indexing and relevance feedback , 2007, Pattern Recognit. Lett..

[17]  Alan F. Smeaton,et al.  Interactive Experiments in Object-Based Retrieval , 2006, CIVR.

[18]  Marc Davis,et al.  Media Streams: an iconic visual language for video annotation , 1993, Proceedings 1993 IEEE Symposium on Visual Languages.

[19]  Peter G. B. Enser,et al.  Visual image retrieval: seeking the alliance of concept-based and content-based paradigms , 2000, J. Inf. Sci..

[20]  Andrew McCallum,et al.  Using Maximum Entropy for Text Classification , 1999 .