Semantic Concept Learning through Massive Internet Video Mining

Semantic concept learning is one of the most challenging problems in video retrieval. The key barrier for semantic concept learning is lack of annotated training data. Internet videos are different from ordinary videos: massive, rich information, customized, non-uniform format, uneven quality, little descriptive text, only a few shots with limited length etc. Therefore, Internet is a potential repository to provide a reliable source for concept learning. In this paper, we focus on the semantic concept learning through known Internet video sources mining. Starting from the video-sharing websites, an automatical graph model generator for concepts relationship learning based on known ontology such as LSCOM, WordNet and ConceptNet is discussed. An automated source discovery method is addressed which prove to be useful in concept detection from the massive Internet videos. Experimental results prove that the addressed method is effective and efficient in semantic concept detection and learning through massive Internet video mining.

[1]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[2]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Thomas S. Huang,et al.  Unifying Keywords and Visual Contents in Image Retrieval , 2002, IEEE Multim..

[4]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[5]  James Ze Wang,et al.  Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  S. Levinson,et al.  A robotic framework for semantic concept learning. , 2004 .

[7]  M. Mattavelli,et al.  Introduction to the special issue on multimedia implementation », IEEE Trans. On Circuits and Systems for Video Technology , 2004 .

[8]  Wei-Ying Ma,et al.  Salient Feature Selection for Visual Concept Learning , 2005, PCM.

[9]  Wei-Ying Ma,et al.  2D Conditional Random Fields for Web information extraction , 2005, ICML.

[10]  Hanna M. Wallach,et al.  Topic modeling: beyond bag-of-words , 2006, ICML.

[11]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[12]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Alberto Del Bimbo,et al.  Semantic adaptation of sport videos with user-centred performance analysis , 2006, IEEE Transactions on Multimedia.

[14]  Shih-Fu Chang,et al.  Columbia University's semantic video search engine , 2007, CIVR '07.

[15]  Chong-Wah Ngo,et al.  Evaluating bag-of-visual-words representations in scene classification , 2007, MIR '07.

[16]  Arif Ghafoor,et al.  Semantic Analysis of Biological Imaging Data: Challenges and Opportunities , 2007, Int. J. Semantic Comput..

[17]  Jean-Marc Odobez,et al.  A Thousand Words in a Scene , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Zhi-Hua Zhou,et al.  Exploiting Image Contents in Web Search , 2007, IJCAI.

[19]  Jay Yagnik,et al.  Learning people annotation from the web via consistency learning , 2007, MIR '07.

[20]  Jiebo Luo,et al.  Large-scale multimodal semantic concept detection for consumer video , 2007, MIR '07.

[21]  Mohamed S. Kamel,et al.  A Text Classification Framework with a Local Feature Ranking for Learning Social Networks , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[22]  Xian-Sheng Hua,et al.  Multi-modality web video categorization , 2007, MIR '07.

[23]  Dong Wang,et al.  Video diver: generic video indexing with diverse features , 2007, MIR '07.

[24]  Alexander G. Hauptmann,et al.  Discriminative Fields for Modeling Semantic Concepts in Video , 2007, RIAO.

[25]  Craig A. Knoblock,et al.  Learning Semantic Definitions of Online Information Sources , 2007, J. Artif. Intell. Res..

[26]  Jiangchuan Liu,et al.  Understanding the Characteristics of Internet Short Video Sharing: YouTube as a Case Study , 2007, ArXiv.

[27]  Shuicheng Yan,et al.  Local Word Bag Model for Text Categorization , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[28]  Gustavo Carneiro,et al.  Supervised Learning of Semantic Classes for Image Annotation and Retrieval , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Yang Yu,et al.  A framework for modeling positive class expansion with single snapshot , 2010, Knowledge and Information Systems.