Concept-concept association information integration and multi-model collaboration for multimedia semantic concept detection

The recent development of the digital camera technology and the popularity of social network websites such as Facebook and Youtube have created huge amounts of multimedia data. Multimedia information is ubiquitous and essential in many applications. In order to fill the gap between data and application requirements (or the so-called semantic gap), advanced methods and tools are needed to automatically mine and annotate high-level concepts to assist in associating the low-level features to the high-level concepts directly. It has been shown that concept-concept association can be effective in bridging the semantic gap in multimedia data. In this paper, a concept-concept association information integration and multi-model collaboration framework is proposed to enhance high-level semantic concept detection from multimedia data. Several experiments are conducted and the comparison results demonstrate that the proposed framework outperforms those approaches in the comparison in terms of the Mean Average Precision (MAP) values.

[1]  Chong-Wah Ngo,et al.  Fast Semantic Diffusion for Large-Scale Context-Based Image and Video Annotation , 2012, IEEE Transactions on Image Processing.

[2]  Serge J. Belongie,et al.  Object categorization using co-occurrence, location and appearance , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Gang Hua,et al.  Semantic Model Vectors for Complex Video Event Recognition , 2012, IEEE Transactions on Multimedia.

[4]  Benoit Huet,et al.  An ontology-based evidential framework for video indexing using high-level multimodal fusion , 2011, Multimedia Tools and Applications.

[5]  John R. Smith,et al.  Multimedia semantic indexing using model vectors , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[6]  K. Ramchandran,et al.  A factor graph framework for semantic indexing and retrieval in video , 2000, 2000 Proceedings Workshop on Content-based Access of Image and Video Libraries.

[7]  Yiannis Kompatsiaris,et al.  Introduction to the special issue on image and video retrieval: theory and applications , 2010, Multimedia Tools and Applications.

[8]  Min Chen,et al.  Video Semantic Event/Concept Detection Using a Subspace-Based Multimedia Data Mining Framework , 2008, IEEE Transactions on Multimedia.

[9]  Tao Mei,et al.  Building a comprehensive ontology to refine video concept detection , 2007, MIR '07.

[10]  Antonio Torralba,et al.  Contextual Priming for Object Detection , 2003, International Journal of Computer Vision.

[11]  Michel Verleysen,et al.  Assessment of probability density estimation methods: Parzen window and finite Gaussian mixtures , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[12]  Shih-Fu Chang,et al.  Active Context-Based Concept Fusionwith Partial User Labels , 2006, 2006 International Conference on Image Processing.

[13]  Meng Wang,et al.  Correlative Linear Neighborhood Propagation for Video Annotation , 2009, IEEE Trans. Syst. Man Cybern. Part B.

[14]  S. Ullman,et al.  Spatial Context in Recognition , 1996, Perception.

[15]  Daphne Koller,et al.  Learning Spatial Context: Using Stuff to Find Things , 2008, ECCV.

[16]  Mei-Ling Shyu,et al.  Model-driven collaboration and information integration for enhancing video semantic concept detection , 2012, 2012 IEEE 13th International Conference on Information Reuse & Integration (IRI).

[17]  Mei-Ling Shyu,et al.  Leveraging Concept Association Network for Multimedia Rare Concept Mining and Retrieval , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[18]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[19]  Stephen Gould,et al.  Multi-Class Segmentation with Relative Location Prior , 2008, International Journal of Computer Vision.

[20]  Brendan J. Frey,et al.  Probabilistic multimedia objects (multijects): a novel approach to video indexing and retrieval in multimedia systems , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[21]  Ashutosh Saxena,et al.  Cascaded Classification Models: Combining Models for Holistic Scene Understanding , 2008, NIPS.

[22]  Chong-Wah Ngo,et al.  Domain adaptive semantic diffusion for large scale context-based video annotation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[23]  Shu-Ching Chen,et al.  Correlation-Based Video Semantic Concept Detection Using Multiple Correspondence Analysis , 2008, 2008 Tenth IEEE International Symposium on Multimedia.

[24]  Ramakrishnan Srikant,et al.  Fast Algorithms for Mining Association Rules in Large Databases , 1994, VLDB.

[25]  Chong-Wah Ngo,et al.  Selection of Concept Detectors for Video Search by Ontology-Enriched Semantic Spaces , 2008, IEEE Transactions on Multimedia.

[26]  Adel M. Alimi,et al.  A fuzzy ontology: based framework for reasoning in visual video content analysis and indexing , 2011, MDMKDD '11.

[27]  Andrea Vedaldi,et al.  Objects in Context , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[28]  Antonio Torralba,et al.  Contextual Models for Object Detection Using Boosted Random Fields , 2004, NIPS.

[29]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[30]  John R. Smith,et al.  Large-scale concept ontology for multimedia , 2006, IEEE MultiMedia.

[31]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[32]  Chao Chen,et al.  Weighted Subspace Filtering and Ranking Algorithms for Video Concept Retrieval , 2011, IEEE MultiMedia.

[33]  Everton Alvares Cherman,et al.  Incorporating label dependency into the binary relevance framework for multi-label classification , 2012, Expert Syst. Appl..

[34]  Yi-Hsuan Yang,et al.  Video search reranking via online ordinal reranking , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[35]  Alberto Del Bimbo,et al.  Video Annotation and Retrieval Using Ontologies and Rule Learning , 2010, IEEE MultiMedia.

[36]  Paul Over,et al.  Evaluation campaigns and TRECVid , 2006, MIR '06.

[37]  Douglas B. Terry,et al.  Using collaborative filtering to weave an information tapestry , 1992, CACM.

[38]  Chao Chen,et al.  Utilization of Co-occurrence Relationships between Semantic Concepts in Re-ranking for Information Retrieval , 2011, 2011 IEEE International Symposium on Multimedia.

[39]  Mubarak Shah,et al.  Improving Semantic Concept Detection and Retrieval using Contextual Estimates , 2007, 2007 IEEE International Conference on Multimedia and Expo.