Automatic event-based indexing of multimedia content using a joint content-event model

In this paper a joint content-event model for the automatic indexing of multimedia content with events is proposed. This model treats events as first class entities and provides a referencing mechanism for automatically linking event elements (represented using the event part of the model) with content segments (described using the content part of the model). The emphasis of the paper is on this mechanism, which uses trained concept detectors to represent content segments with model vectors, and the subclass discriminant analysis algorithm to derive a discriminant subspace facilitating the indexing of content segments with event elements. The use of this referencing mechanism for associating multimedia content with five sport events is demonstrated on the MediaMill dataset.

[1]  Yiannis Kompatsiaris,et al.  MESH participation to TRECVID2008 HLFE , 2008, TRECVID.

[2]  Wen-Huang Cheng,et al.  Semantic-event based analysis and segmentation of wedding ceremony videos , 2007, MIR '07.

[3]  Jeffrey M. Zacks,et al.  Human brain activity time-locked to perceptual event boundaries , 2001, Nature Neuroscience.

[4]  Aleix M. Martínez,et al.  Subclass discriminant analysis , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Pradeep K. Atrey,et al.  A hierarchical model for representation of events in multimedia observation systems , 2009, EiMM '09.

[6]  Konstantinos N. Plataniotis,et al.  Face recognition using kernel direct discriminant analysis algorithms , 2003, IEEE Trans. Neural Networks.

[7]  Xiaofeng Wang,et al.  Ice hockey shot event modeling with mixture hidden Markov model , 2009, EiMM '09.

[8]  Ramesh Jain,et al.  Toward a Common Event Model for Multimedia Applications , 2007, IEEE MultiMedia.

[9]  John R. Smith,et al.  Multimedia semantic indexing using model vectors , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[10]  Steffen Staab,et al.  F--a model of events based on the foundational ontology dolce+DnS ultralight , 2009, K-CAP '09.

[11]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[12]  David G. Stork,et al.  Pattern Classification , 1973 .

[13]  Alberto Del Bimbo,et al.  Video event classification using string kernels , 2010, Multimedia Tools and Applications.

[14]  Ramakant Nevatia,et al.  VERL: An Ontology Framework for Representing and Annotating Video Events , 2005, IEEE Multim..

[15]  Anthony G. Cohn,et al.  A Spatial Logic based on Regions and Connection , 1992, KR.

[16]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[17]  Marcel Worring,et al.  The challenge problem for automated detection of 101 semantic concepts in multimedia , 2006, MM '06.

[18]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Yiannis Kompatsiaris,et al.  Multi-modal scene segmentation using scene transition graphs , 2009, ACM Multimedia.

[20]  Nicola Guarino,et al.  Sweetening Ontologies with DOLCE , 2002, EKAW.

[21]  Yiannis Kompatsiaris,et al.  Gradual transition detection using color coherence and other criteria in a video shot meta-segmentation framework , 2008, 2008 15th IEEE International Conference on Image Processing.

[22]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.