Animated movie activity characterization by image and text information fusion

In the context of animated movie characterization, we present an information fusion approach mixing very different types of data related to the activity within a movie. These data are the features extracted from images, words extracted from the synopses and expert knowledge. The difficulty of this fusion is due to the very different semantic level of these data. The aim of this work is to get a movie activity characterization in order to help the constitution of automatic summary, content based video retrieval system, etc. Two strategies are proposed : a first one aiming at giving a global description of the activity within the movie, and a second one providing a local description of activity. Tests and results are proposed on animated movies from the Annecy International Animation Film Festival.

[1]  Ian H. Witten,et al.  Weka: Practical machine learning tools and techniques with Java implementations , 1999 .

[2]  Changsheng Xu,et al.  Segmentation, categorization, and identification of commercial clips from TV streams using multimodal analysis , 2006, MM '06.

[3]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[4]  Georges Quénot,et al.  Une Approche Conceptuelle pour la Modlisation et la Structuration Smantique des Documents Vidos , 2005 .

[5]  Douglas E. Appelt,et al.  Introduction to Information Extraction , 1999, AI Commun..

[6]  Teresa Gonçalves,et al.  A Preliminary Approach to the Multilabel Classification Problem of Portuguese Juridical Documents , 2003, EPIA.

[7]  Marcel Worring,et al.  Multimodal Video Indexing : A Review of the State-ofthe-art , 2001 .

[8]  Qi Tian,et al.  Probabilistic optimized ranking for multimedia semantic concept detection via RVM , 2008, CIVR '08.

[9]  Denis Lalanne,et al.  DocMIR: An automatic document-based indexing system for meeting retrieval , 2008, Multimedia Tools and Applications.

[10]  Ioannis Pitas,et al.  An anthropocentric description scheme for movies content classification and indexing , 2005, 2005 13th European Signal Processing Conference.

[11]  Liang-Tien Chia,et al.  Semantic Video Indexing and Summarization Using Subtitles , 2004, PCM.

[12]  Bogdan-Emanuel Ionescu.  Caractérisation symbolique de séquences d’images : application aux films d’animation , 2007 .

[13]  Guy Lapalme,et al.  exibum : Un systeme experimental d'extraction d'information bilingue , 1998 .

[14]  Christian Gütl,et al.  Multi-label Text Classification of German Language Medical Documents , 2007, MedInfo.

[15]  Ralph Grishman,et al.  Information Extraction: Techniques and Challenges , 1997, SCIE.

[16]  Ponnuthurai N. Suganthan,et al.  An Accumulation Algorithm for Video Shot Boundary Detection , 2004, Multimedia Tools and Applications.

[17]  Yi-Hsuan Yang,et al.  Keyword-based concept search on consumer photos by web-based kernel function , 2008, ACM Multimedia.

[18]  Cordelia Schmid,et al.  Learning realistic human actions from movies , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Khaled Elleithy.  Advances and Innovations in Systems, Computing Sciences and Software Engineering , 2007 .

[20]  Daniel Dominic Sleator,et al.  Parsing English with a Link Grammar , 1995, IWPT.

[21]  Zhu Liu,et al.  Multimedia content analysis-using both audio and visual clues , 2000, IEEE Signal Process. Mag..